Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

NEURAL NETWORKS IN DATA MINING

1 2
DR. YASHPAL SINGH, ALOK SINGH CHAUHAN
1
Reader, Bundelkhand Institute of Engineering & Technology, Jhansi, India
2
Lecturer, United Institute of Management, Allahabad, India
E-mail: [email protected] , [email protected]

ABSTRACT

Companies have been collecting data for decades, building massive data warehouses in which to store it.
Even though this data is available, very few companies have been able to realize the actual value stored in
it. The question these companies are asking is how to extract this value. The answer is Data mining.
There are many technologies available to data mining practitioners, including Artificial Neural Networks,
Regression, and Decision Trees. Many practitioners are wary of Neural Networks due to their black box
nature, even though they have proven themselves in many situations. This paper is an overview of artificial
neural networks and questions their position as a preferred tool by data mining practitioners.

Keywords: Artificial Neural Network (ANN), neural network topology, Data mining, back propagation
algorithm, Advantages.

1. INTRODUCTION:
Income is a very important socio-economic
Data mining is the term used to describe indicator. If a bank knows a person’s income, they
the process of extracting value from a database. A can offer a higher credit card limit or determine if
data-warehouse is a location where information is they are likely to want information on a home loan
stored. The type of data stored depends largely on or managed investments. Even though this financial
the type of industry and the company. Many institution had the ability to determine a customer’s
companies store every piece of data they have income in two ways, from their credit card
collected, while others are more ruthless in what application, or through regular direct deposits into
they deem to be “important”. their bank account, they did not extract and utilize
Consider the following example of a financial this information.
institution failing to utilize their data-warehouse.
Another example of where this institution has failed
to utilize its data-warehouse is in cross-selling 2. ARTIFICIAL NEURAL NETWORKS:
insurance products (e.g. home, life and motor
vehicle insurance). By using transaction An artificial neural network (ANN), often just
information they may have the ability to determine called a "neural network" (NN), is a mathematical
if a customer is making payments to another model or computational model based on biological
insurance broker. This would enable the institution neural networks, in other words, is an emulation of
to select prospects for their insurance products. biological neural system. It consists of an
These are simple examples of what could be interconnected group of artificial neurons and
achieved using data mining. processes information using a connectionist
Four things are required to data-mine effectively: approach to computation. In most cases an ANN is
high-quality data, the “right” data, an adequate an adaptive system that changes its structure based
sample size and the right tool. There are many tools on external or internal information that flows
available to a data mining practitioner. These through the network during the learning phase.
include decision trees, various types of regression
and neural networks.

37
 Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

2.1 Neural Network Topologies: external teacher, or by the system which

Feedforward neural network: The feedforward contains the neural network (self-supervised).
neural network was the first and arguably simplest • Unsupervised learning or Self-organization in
type of artificial neural network devised. In this which an (output) unit is trained to respond to
network, the information moves in only one clusters of pattern within the input. In this
direction, forward, from the input nodes, through paradigm the system is supposed to discover
the hidden nodes (if any) and to the output nodes. statistically salient features of the input
There are no cycles or loops in the network. The population. Unlike the supervised learning
data processing can extend over multiple (layers of) paradigm, there is no a priori set of categories
units, but no feedback connections are present, that into which the patterns are to be classified;
is, connections extending from outputs of units to rather the system must develop its own
inputs of units in the same layer or previous layers. representation of the input stimuli.
Recurrent network: Recurrent neural networks Reinforcement Learning This type of
that do contain feedback connections. Contrary to learning may be considered as an intermediate
feedforward networks, recurrent neural networks form of the above two types of learning. Here
(RNs) are models with bi-directional data flow. the learning machine does some action on the
While a feedforward network propagates data environment and gets a feedback response
linearly from input to output, RNs also propagate from the environment. The learning system
data from later processing stages to earlier stages. grades its action good (rewarding) or bad
(punishable) based on the environmental
2.2 Training Of Artificial Neural Networks: response and accordingly adjusts its
A neural network has to be configured such that parameters.
the application of a set of inputs produces (either
'direct' or via a relaxation process) the desired set of 3. NEURAL NETWORKS IN DATA MINING:
outputs. Various methods to set the strengths of the
connections exist. One way is to set the weights In more practical terms neural networks
explicitly, using a priori knowledge. Another way is are non-linear statistical data modeling tools. They
to 'train' the neural network by feeding it can be used to model complex relationships
teaching patterns and letting it change its weights between inputs and outputs or to find patterns in
according to some learning rule. We can categorize data. Using neural networks as a tool, data
the learning situations as follows: warehousing firms are harvesting information from
• Supervised learning or Associative learning datasets in the process known as data mining. The
in which the network is trained by providing it difference between these data warehouses and
with input and matching output patterns. These ordinary databases is that there is actual anipulation
input-output pairs can be provided by an and cross-fertilization of the data helping users
makes more informed decisions.

38
 Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

Neural networks essentially comprise three pieces: Prediction, Clustering, Association Rules.
the architecture or model; the learning algorithm; Classification and prediction is a predictive model,
and the activation functions. Neural networks are but clustering and association rules are descriptive
programmed or “trained” to “. . . store, recognize, models.
and associatively retrieve patterns or database The most common action in data mining is
entries; to solve combinatorial optimization classification. It recognizes patterns that describe
problems; to filter noise from measurement data; to the group to which an item belongs. It does this by
control ill-defined problems; in summary, to examining existing items that already have been
estimate sampled functions when we do not know classified and inferring a set of rules. Similar to
the form of the functions.” It is precisely these two classification is clustering. The major difference
abilities (pattern recognition and function being that no groups have been predefined.
estimation) which make artificial neural networks Prediction is the construction and use of a model to
(ANN) so prevalent a utility in data mining. As data assess the class of an unlabeled object or to assess
sets grow to massive sizes, the need for automated the value or value ranges of a given object is likely
processing becomes clear. With their “model-free” to have. The next application is forecasting. This is
estimators and their dual nature, neural networks different from predictions because it estimates the
serve data mining in a myriad of ways. future value of continuous variables based on
Data mining is the business of answering questions patterns within the data. Neural networks,
that you’ve not asked yet. Data mining reaches depending on the architecture, provide associations,
deep into databases. Data mining tasks can be classifications, clusters, prediction and forecasting
classified into two categories: Descriptive and to the data mining industry.
predictive data mining. Descriptive data mining Financial forecasting is of considerable practical
provides information to understand what is interest. Due to neural networks can mine valuable
happening inside the data without a predetermined information from a mass of history information and
idea. Predictive data mining allows the user to be efficiently used in financial areas, so the
submit records with unknown field values, and the applications of neural networks to financial
system will guess the unknown values based on forecasting have been very popular over the last
previous patterns discovered form the database. few years. Some researches show that neural
Data mining models can be categorized according networks performed better than conventional
to the tasks they perform: Classification and statistical approaches in financial forecasting and

39
 Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

are an excellent data mining tool. In data information on associations, classifications,

warehouses, neural networks are just one of the clusters, and forecasting. The back propagation
tools used in data mining. ANNs are used to find algorithm performs learning on a feed-forward
patterns in the data and to infer rules from them. neural network.
Neural networks are useful in providing

3.1. Feedforward Neural Network:

One of the simplest feed forward neural the previous layer. There are connections between
networks (FFNN), such as in Figure, consists of the PEs in each layer that have a weight (parameter)
three layers: an input layer, hidden layer and output associated with them. This weight is adjusted
layer. In each layer there are one or more during training. Information only travels in the
processing elements (PEs). PEs is meant to forward direction through the network - there are
simulate the neurons in the brain and this is why no feedback loops.
they are often referred to as neurons or nodes. A PE
receives inputs from either the outside world or

The simplified process for training a FFNN is as 3.2. The Back Propagation Algorithm:
follows: Backpropagation, or propagation of error, is a
1. Input data is presented to the network and common method of teaching artificial neural
propagated through the network until it reaches networks how to perform a given task.The back
the output layer. This forward process produces propagation algorithm is used in layered feed-
a predicted output. forward ANNs. This means that the artificial
2. The predicted output is subtracted from the neurons are organized in layers, and send their
actual output and an error value for the signals “forward”, and then the errors are
networks is calculated. propagated backwards. The back propagation
3. The neural network then uses supervised algorithm uses supervised learning, which means
learning, which in most cases is back that we provide the algorithm with examples of the
propagation, to train the network. Back inputs and outputs we want the network to
propagation is a learning algorithm for compute, and then the error (difference between
adjusting the weights. It starts with the weights actual and expected results) is calculated. The idea
between the output layer PE’s and the last of the back propagation algorithm is to reduce this
hidden layer PE’s and works backwards error, until the ANN learns the training data.
through the network. Summary of the technique:
4. Once back propagation has finished, the 1. Present a training sample to the neural
forward process starts again, and this cycle is network.
continued until the error between predicted and 2. Compare the network's output to the desired
actual outputs is minimized. output from that sample. Calculate the error in
each output neuron.

40
 Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

3. For each neuron, calculate what the output 5. Assign "blame" for the local error to neurons at
should have been, and a scaling factor, how the previous level, giving greater responsibility
much lower or higher the output must be to neurons connected by stronger weights.
adjusted to match the desired output. This is 6. Repeat the steps above on the neurons at the
the local error. previous level, using each one's "blame" as its
4. Adjust the weights of each neuron to lower the error..
local error.
Actual Algorithm: fraud detection, telecommunications, medicine,
1. Initialize the weights in the network (often marketing, bankruptcy prediction, insurance, the
randomly) list goes on. The following are examples of where
2. repeat neural networks have been used.
* for each example e in the training set do Accounting
1. O = neural-net-output(network, e) ; Identifying tax fraud
forward pass Enhancing auditing by finding irregularities
2. T = teacher output for e Finance
3. Calculate error (T - O) at the output Signature and bank note verification
units Risk Management
4. Compute delta_wi for all weights Foreign exchange rate forecasting
from hidden layer to output layer ; Bankruptcy prediction
backward pass Customer credit scoring
5. Compute delta_wi for all weights Credit card approval and fraud detection
from input layer to hidden layer ; Forecasting economic turning points
backward pass continued Bond rating and trading
6. Update the weights in the network Loan approvals
* end Economic and financial forecasting
3. until all examples classified correctly or Marketing
stopping criterion satisfied Classification of consumer spending pattern
4. return(network) New product analysis
Identification of customer characteristics
Sale forecasts
4. REVIEW OF LITERATURE REPORTING Human resources
NEURAL NETWORK PERFORMANCE: Predicting employee’s performance and
There are numerous examples of commercial behavior
applications for neural networks. These include; Determining personnel resource requirements

5. ADVANTAGES OF NEURAL 6. DESIGN PROBLEMS:

NETWORKS: ƒ There are no general methods to determine the
1. High Accuracy: Neural networks are able to optimal number of neurones necessary for
approximate complex non-linear mappings solving any problem.
2. Noise Tolerance: Neural networks are very
flexible with respect to incomplete, missing ƒ It is difficult to select a training data set which
and noisy data. fully describes the problem to be solved.
3. Independence from prior assumptions: Neural
networks do not make a priori assumptions 7. SOLUTIONS TO IMPROVE ANN
about the distribution of the data, or the form PERFORMANCE:
of interactions between factors. ƒ Designing Neural Networks using Genetic
4. Ease of maintenance: Neural networks can be Algorithms
updated with fresh data, making them useful ƒ Neuro-Fuzzy Systems
for dynamic environments.
5. Neural networks can be implemented in 8. CONCLUSION:
parallel hardware There is rarely one right tool to use in data mining;
6. When an element of the neural network fails, it it is a question as to what is available and what
can continue without any problem by their gives the “best” results. Many articles, in addition
parallel nature. to those mentioned in this paper, consider neural
networks to be a promising data mining tool.

41
 Journal of Theoretical and Applied Information Technology

© 2005 - 2009 JATIT. All rights reserved.

www.jatit.org

Artificial Neural Networks offer qualitative researchers use them, in particular those with
methods for business and economic systems that statistical backgrounds. Thus, neural networks are
traditional quantitative tools in statistics and becoming very popular with data mining
econometrics cannot quantify due to the complexity practitioners, particularly in medical research,
in translating the systems into precise mathematical finance and marketing. This is because they have
functions. Hence, the use of neural networks indata proven their predictive power through comparison
mining is a promising field of research especially with other statistical techniques using real data sets.
given the ready availability of large mass of data Due to design problems neural systems need further
sets and the reported ability of neural networks to research before they are widely accepted in
detect and assimilate relationships between a large industry. As software companies develop more
numbers of variables. sophisticated models with user-friendly interfaces
In most cases neural networks perform as well or the attraction to neural networks will continue to
better than the traditional statistical techniques to grow.
which they are compared. Resistance to using these
“black boxes” is gradually diminishing as more

REFERENCES

[1] Agrawal, R., Imielinski, T., Swami, A.,

“Database Mining: A Performance
Perspective”, IEEE Transactions on
Knowledge and Data Engineering, pp. 914-
925, December 1993

[2] Berry, J. A., Lindoff, G., Data Mining

Techniques, Wiley Computer Publishing,
1997 (ISBN 0-471-17980-9).

[3] Berson, “Data Warehousing, Data-Mining &

OLAP”, TMH

[4] Bhavani,Thura-is-ingham, “Data-mining

Technologies,Techniques tools & Trends”,
CRC Press

[5] Bradley, I., Introduction to Neural Networks,

Multinet Systems Pty Ltd 1997.

[6] Fayyad, Usama, Ramakrishna “ Evolving Data

mining into solutions for Insights”,
communications of the ACM 45, no. 8

[7] Fausett, Laurene (1994), Fundamentals of

Neural Networks: Architectures, Algorithms
and Applications, Prentice-Hall, New Jersey,
USA.

[8] Haykin, S., Neural Networks, Prentice Hall

International Inc., 1999

[9] Khajanchi, Amit, Artificial Neural Networks:

The next intelligence

[10] Zurada J.M., “An introduction to artificial

neural networks systems”, St. Paul: West
Publishing (1992)

42