Neural Networks with Java: neural net overview

choose a section

What are neural nets?

The biological model: The human brain
The components of a neural net
What they can and where they fail

Types of neural nets

Perceptron
Multi-Layer-Perceptron
Backpropagation Net
Hopfield Net
Kohonen Feature Map

The learning process

What means "learning" refering to neural nets?
Supervised and unsupervised learning
Forwardpropagation
Backpropagation
Selforganization
 Neural Networks with Java: neural net overview

jump directly to

What are neural nets?

The biological model: The human brain
The components of a neural net
What they can and where they fail

What are neural nets?

A neural net is an artificial representation of the human brain that tries to simulate its learning
process.
The term "artificial" means that neural nets are implemented in computer programs that are able
to handle the large number of neccessary calculations during the learning process.

To show where neural nets have their origin, let's have a look at the biological model: the human
brain.

The biological model: The human brain

The human brain consists of a large number (more than a billion) of neural cells that process
informations. Each cell works like a simple processor and only the massive interaction between
all cells and their parallel processing makes the brain's abilities possible.

Below you see a sketch of such a neural cell, called a neuron:

 Neural Networks with Java: neural net overview

Structure of a neural cell in the human brain

As the figure indicates, a neuron consists of a core, dendrites for incoming information and an
axon with dendrites for outgoing information that is passed to connected neurons.
Information is transported between neurons in form of electrical stimulations along the dendrites.
Incoming informations that reach the neuron's dendrites is added up and then delivered along the
neuron's axon to the dendrites at its end, where the information is passed to other neurons if the
stimulation has exceeded a certain threshold. In this case, the neuron is said to be activated.
If the incoming stimulation had been too low, the information will not be transported any further.
In this case, the neuron is said to be inhibited.

The connections between the neurons are adaptive, what means that the connection structure is
changing dynamically. It is commonly acknowledged that the learning ability of the human brain
is based on this adaptation.

The components of a neural net

Generally spoken, there are many different types of neural nets, but they all have nearly the same
components.
If one wants to simulate the human brain using a neural net, it is obviously that some drastic
simplifications have to be made:
First of all, it is impossible to "copy" the true parallel processing of all neural cells. Although
there are computers that have the ability of parallel processing, the large number of processors
that would be necessary to realize it can't be afforded by today's hardware.
Another limitation is that a computer's internal structure can't be changed while performing any
tasks.
Neural Networks with Java: neural net overview

And how to implement electrical stimulations in a computer program?

These facts lead to an idealized model for simulation purposes.

Like the human brain, a neural net also consists of neurons and connections between them. The
neurons are transporting incoming information on their outgoing connections to other neurons. In
neural net terms these connections are called weights. The "electrical" information is simulated
with specific values stored in those weights.
By simply changing these weight values the changing of the connection structure can also be
simulated.

The following figure shows an idealized neuron of a neural net.

Structure of a neuron in a neural net

As you can see, an artificial neuron looks similar to a biological neural cell. And it works in the
same way.
Information (called the input) is sent to the neuron on its incoming weights. This input is
processed by a propagation function that adds up the values of all incoming weights.
The resulting value is compared with a certain threshold value by the neuron's activation function.
If the input exceeds the threshold value, the neuron will be activated, otherwise it will be
inhibited.
If activated, the neuron sends an output on its outgoing weights to all connected neurons and so
on.

In a neural net, the neurons are grouped in layers, called neuron layers. Usually each neuron of
one layer is connected to all neurons of the preceding and the following layer (except the input
layer and the output layer of the net).
The information given to a neural net is propagated layer-by-layer from input layer to output layer
through either none, one or more hidden layers. Depending on the learning algorithm, it is also
possible that information is propagated backwards through the net.
 Neural Networks with Java: neural net overview

The following figure shows a neural net with three neuron layers.

Neural net with three neuron layers

Note that this is not the general structure of a neural net. For example, some neural net types have
no hidden layers or the neurons in a layer are arranged as a matrix.
What's common to all neural net types is the presence of at least one weight matrix, the
connections between two neuron layers.

Next, let's see what neural nets are useful for.

What they can and where they fail

Neural nets are being constructed to solve problems that can't be solved using conventional
algorithms.
Such problems are usually optimization or classification problems.
Neural Networks with Java: neural net overview

The different problem domains where neural nets may be used are:

❍ pattern association
❍ pattern classification
❍ regularity detection
❍ image processing
❍ speech analysis
❍ optimization problems
❍ robot steering
❍ processing of inaccurate or incomplete inputs
❍ quality assurance
❍ stock market forecasting
❍ simulation
❍ ...

There are many different neural net types with each having special properties, so each problem
domain has its own net type (see Types of neural nets for a more detailed description).
Generally it can be said that neural nets are very flexible systems for problem solving purposes.
One ability should be mentioned explicitely: the error tolerance of neural networks. That means,
if a neural net had been trained for a specific problem, it will be able to recall correct results, even
if the problem to be solved is not exactly the same as the already learned one. For example,
suppose a neural net had been trained to recognize human speech. During the learning process, a
certain person has to pronounce some words, which are learned by the net. Then, if trained
correctly, the neural net should be able to recognize those words spoken by another person, too.

But all that glitters ain't gold. Although neural nets are able to find solutions for difficult problems
as listed above, the results can't be guaranteed to be perfect or even correct. They are just
approximations of a desired solution and a certain error is always present.
Additionaly, there exist problems that can't be correctly solved by neural nets. An example on
pattern recognition should settle this:
If you meet a person you saw earlier in your life, you usually will recognize him/her the second
time, even if he/she doesn't look the same as at your first encounter.
Suppose now, you trained a neural net with a photograph of that person, this image will surely be
recognized by the net. But if you add heavy noise to the picture or rotate it to some degree, the
recognition will probably fail.

Surely, nobody would ever use a neural network in a sorting algorithm, for there exist much better
and faster algorithms, but in problem domains, as those mentioned above, neural nets are always a
good alternative to existing algorithms and definitely worth a try.
Neural Networks with Java: neural net overview
 Neural Networks with Java: neural net overview

jump directly to

Types of neural nets

Perceptron
Multi-Layer-Perceptron
Backpropagation Net
Hopfield Net
Kohonen Feature Map

Types of Neural Nets

As mentioned before, several types of neural nets exist.

They can be distinguished by
their type (feedforward or feedback),
their structure
and the learning algorithm they use.
The type of a neural net indicates, if the neurons of one of the net's layers may be connected
among each other. Feedforward neural nets allow only neuron connections between two different
layers, while nets of the feedback type have also connections between neurons of the same layer.

In this section, a selection of neural nets will be described.

Perceptron

The Perceptron was first introduced by F. Rosenblatt in 1958.

It is a very simple neural net type with two neuron layers that accepts only binary input and output
values (0 or 1). The learning process is supervised and the net is able to solve basic logical
operations like AND or OR. It is also used for pattern classification purposes.
More complicated logical operations (like the XOR problem) cannot be solved by a Perceptron.

Perceptron characteristics
 Neural Networks with Java: neural net overview

sample structure

type feedforward
neuron layers 1 input layer
1 output layer
input value binary
types
activation hard limiter
function
learning method supervised
learning Hebb learning rule
algorithm
mainly used in simple logical operations
pattern classification

Multi-Layer-Perceptron

The Multi-Layer-Perceptron was first introduced by M. Minsky and S. Papert in 1969.

It is an extended Perceptron and has one ore more hidden neuron layers between its input and
output layers.
Due to its extended structure, a Multi-Layer-Perceptron is able to solve every logical operation,
including the XOR problem.
Neural Networks with Java: neural net overview

Multi-Layer-Perceptron characteristics
sample
structure

type feedforward
neuron layers 1 input layer
1 or more hidden layers
1 output layer
input value binary
types
activation hard limiter / sigmoid
function
learning supervised
method
learning delta learning rule
algorithm backpropagation (mostly used)
mainly used in complex logical operations
pattern classification
 Neural Networks with Java: neural net overview

Backpropagation Net

The Backpropagation Net was first introduced by G.E. Hinton, E. Rumelhart and R.J. Williams in
1986
and is one of the most powerful neural net types.
It has the same structure as the Multi-Layer-Perceptron and uses the backpropagation learning
algorithm.

Backpropagation Net characteristics

sample
structure

type feedforward
neuron layers 1 input layer
1 or more hidden layers
1 output layer
input value binary
types
 Neural Networks with Java: neural net overview

activation sigmoid
function
learning supervised
method
learning backpropagation
algorithm
mainly used in complex logical operations
pattern classification
speech analysis

Hopfield Net

The Hopfield Net was first introduced by physicist J.J. Hopfield in 1982 and belongs to neural net
types which are called "thermodynamical models".
It consists of a set of neurons, where each neuron is connected to each other neuron. There is no
differentiation between input and output neurons.
The main application of a Hopfield Net is the storage and recognition of patterns, e.g. image files.

Hopfield Net characteristics

sample
structure

http://rfhs8012.fh-regensburg.de/~saj39122/jfroehl/diplom/e-12-text.html (5 of 8) [1/1/2002 12:45:09 PM]

 Neural Networks with Java: neural net overview

type feedback
neuron 1 matrix
layers
input value binary
types
activation signum / hard limiter
function
learning unsupervised
method
learning delta learning rule
algorithm simulated annealing (mostly used)
mainly pattern association
used in optimization problems

Kohonen Feature Map

The Kohonen Feature Map was first introduced by finnish professor Teuvo Kohonen (University
of Helsinki) in 1982.
It is probably the most useful neural net type, if the learning process of the human brain shall be
simulated. The "heart" of this type is the feature map, a neuron layer where neurons are
organizing themselves according to certain input values.
The type of this neural net is both feedforward (input layer to feature map) and feedback (feature
map).
(A Kohonen Feature Map is used in the sample applet)

Kohonen Feature Map characteristics

http://rfhs8012.fh-regensburg.de/~saj39122/jfroehl/diplom/e-12-text.html (6 of 8) [1/1/2002 12:45:09 PM]

Neural Networks with Java: neural net overview

sample
structure

type feedforward / feedback

neuron 1 input layer
layers 1 map layer
input binary, real
value
types
activation sigmoid
function
learning unsupervised
method
learning selforganization
algorithm
mainly pattern classification
used in optimization problems
simulation
 Neural Networks with Java: neural net overview

Types of neural nets bottom of page

jump directly to

The learning process

What does "learning" mean refering to neural nets?
Supervised and unsupervised learning
Forwardpropagation
Backpropagation
Selforganization

The learning process

This section describes the learning ability of neural networks.

First, the term learning is explained, followed by an overview of specific learning algorithms for neural nets.

What does "learning" mean refering to neural nets?

In the human brain, information is passed between the neurons in form of electrical stimulation along the dendrites. If a certain
amount of stimulation is received by a neuron, it generates an output to all other connected neurons and so information takes its
way to its destination where some reaction will occur. If the incoming stimulation is too low, no output is generated by the neuron
and the information's further transport will be blocked.

Explaining how the human brain learns certain things is quite difficult and nobody knows it exactly.
It is supposed that during the learning process the connection structure among the neurons is changed, so that certain stimulations
are only accepted by certain neurons. This means, there exist firm connections between the neural cells that once have learned a
specific fact, enabling the fast recall of this information.
If some related information is acquired later, the same neural cells are stimulated and will adapt their connection structure
according to this new information.
On the other hand, if a specific information isn't recalled for a long time, the established connection structure between the
responsible neural cells will get more "weak". This had happened if someone "forgot" a once learned fact or can only remember it
vaguely.

As mentioned before, neural nets try to simulate the human brain's ability to learn. That is, the artificial neural net is also made of
neurons and dendrites. Unlike the biological model, a neural net has an unchangeable structure, built of a specified number of
neurons and a specified number of connections between them (called "weights"), which have certain values.
What changes during the learning process are the values of those weights. Compared to the original this means:
Incoming information "stimulates" (exceeds a specified threshold value of) certain neurons that pass the information to connected
neurons or prevent further transportation along the weighted connections. The value of a weight will be increased if information
should be transported and decreased if not.

While learning different inputs, the weight values are changed dynamically until their values are balanced, so each input will lead
to the desired output.
The training of a neural net results in a matrix that holds the weight values between the neurons. Once a neural net had been trained
correctly, it will probably be able to find the desired output to a given input that had been learned, by using these matrix values.
I said "probably". That is sad but true, for it can't be guaranteed that a neural net will recall the correct results in any case.
Very often there is a certain error left after the learning process, so the generated output is only a good approximation to the perfect
output in most cases.
 Neural Networks with Java: neural net overview

The following sections introduce several learning algorithms for neural networks.

Supervised and unsupervised learning

The learning algorithm of a neural network can either be supervised or unsupervised.

A neural net is said to learn supervised, if the desired output is already known.
Example: pattern association
Suppose, a neural net shall learn to associate the following pairs of patterns. The input patterns are decimal numbers, each
represented in a sequence of bits. The target patterns are given in form of binary values of the decimal numbers:

input pattern target pattern

0001 001
0010 010
0100 011
1000 100

While learning, one of the input patterns is given to the net's input layer. This pattern is propagated through the net (independent of
its structure) to the net's output layer. The output layer generates an output pattern which is then compared to the target pattern.
Depending on the difference between output and target, an error value is computed.
This output error indicates the net's learning effort, which can be controlled by the "imaginary supervisor". The greater the
computed error value is, the more the weight values will be changed.

Neural nets that learn unsupervised have no such target outputs.

It can't be determined what the result of the learning process will look like.
During the learning process, the units (weight values) of such a neural net are "arranged" inside a certain range, depending on
given input values. The goal is to group similar units close together in certain areas of the value range.
This effect can be used efficiently for pattern classification purposes.
See Selforganization for details.

Forwardpropagation

Forwardpropagation is a supervised learning algorithm and describes the "flow of information" through a neural net from its input
layer to its output layer.

The algorithm works as follows:

1. Set all weights to random values ranging from -1.0 to +1.0

2. Set an input pattern (binary values) to the neurons of the net's input layer
3. Activate each neuron of the following layer:
Multiply the weight values of the connections leading to this neuron with the output values of the preceding neurons
Add up these values
Pass the result to an activation function, which computes the output value of this neuron
7. Repeat this until the output layer is reached
8. Compare the calculated output pattern to the desired target pattern and compute an error value
9. Change all weights by adding the error value to the (old) weight values
10. Go to step 2
11. The algorithm ends, if all output patterns match their target patterns
Neural Networks with Java: neural net overview

Example:
Suppose you have the following 2-layered Perceptron:

Patterns to be learned:

input target
01 0
11 1

First, the weight values are set to random values (0.35 and 0.81).
The learning rate of the net is set to 0.25.
Next, the values of the first input pattern (0 1) are set to the neurons of the input layer (the output of the input layer is the
same as its input).
The neurons in the following layer (only one neuron in the output layer) are activated:

Input 1 of output neuron: 0 * 0.35 = 0

Input 2 of output neuron: 1 * 0.81 = 0.81
Add the inputs: 0 + 0.81 = 0.81 (= output)
Compute an error value by
subtracting output from target: 0 - 0.81 = -0.81
Value for changing weight 1: 0.25 * 0 * (-0.81) = 0 (0.25 = learning
rate)
Value for changing weight 2: 0.25 * 1 * (-0.81) = -0.2025
Change weight 1: 0.35 + 0 = 0.35 (not changed)
Change weight 2: 0.81 + (-0.2025) = 0.6075

Now that the weights are changed, the second input pattern (1 1) is set to the input layer's neurons and the activation of the
output neuron is performed again, now with the new weight values:

Input 1 of output neuron: 1 * 0.35 = 0.35

Input 2 of output neuron: 1 * 0.6075 = 0.6075
Add the inputs: 0.35 + 0.6075 = 0.9575 (= output)
Compute an error value by
subtracting output from target: 1 - 0.9575 = 0.0425
Value for changing weight 1: 0.25 * 1 * 0.0425 = 0.010625
Value for changing weight 2: 0.25 * 1 * 0.0425 = 0.010625
Change weight 1: 0.35 + 0.010625 = 0.360625
Change weight 2: 0.6075 + 0.010625 = 0.618125

That was one learning step. Each input pattern had been propagated through the net and the weight values were changed.
The error of the net can now be calculated by adding up the squared values of the output errors of each pattern:

Compute the net error: (-0.81)2 + (0.0425)2 = 0.65790625

By performing this procedure repeatedly, this error value gets smaller and smaller.
 Neural Networks with Java: neural net overview

The algorithm is successfully finished, if the net error is zero (perfect) or approximately zero.

Backpropagation

Backpropagation is a supervised learning algorithm and is mainly used by Multi-Layer-Perceptrons to change the weights
connected to the net's hidden neuron layer(s).

The backpropagation algorithm uses a computed output error to change the weight values in backward direction.
To get this net error, a forwardpropagation phase must have been done before. While propagating in forward direction, the neurons
are being activated using the sigmoid activation function.

The formula of sigmoid activation is:

1
f(x) = ---------
1 + e-input

The algorithm works as follows:

1. Perform the forwardpropagation phase for an input pattern and calculate the output error
2. Change all weight values of each weight matrix using the formula
weight(old) + learning rate * output error * output(neurons i) * output(neurons i+1) * ( 1 - output(neurons i+1) )
4. Go to step 1
5. The algorithm ends, if all output patterns match their target patterns

Example:
Suppose you have the following 3-layered Multi-Layer-Perceptron:

Patterns to be learned:

input target
01 0
11 1

First, the weight values are set to random values: 0.62, 0.42, 0.55, -0.17 for weight matrix 1 and 0.35, 0.81 for weight
matrix 2.
The learning rate of the net is set to 0.25.
Next, the values of the first input pattern (0 1) are set to the neurons of the input layer (the output of the input layer is the
same as its input).
Neural Networks with Java: neural net overview

The neurons in the hidden layer are activated:

Input of hidden neuron 1: 0 * 0.62 + 1 * 0.55 = 0.55

Input of hidden neuron 2: 0 * 0.42 + 1 * (-0.17) = -0.17
Output of hidden neuron 1: 1 / ( 1 + exp(-0.55) ) = 0.634135591
Output of hidden neuron 2: 1 / ( 1 + exp(+0.17) ) = 0.457602059

The neurons in the output layer are activated:

Input of output neuron: 0.634135591 * 0.35 + 0.457602059 * 0.81 =

0.592605124
Output of output neuron: 1 / ( 1 + exp(-0.592605124) ) = 0.643962658
Compute an error value by
subtracting output from target: 0 - 0.643962658 = -0.643962658

Now that we got the output error, let's do the backpropagation.

We start with changing the weights in weight matrix 2:

Value for changing weight 1: 0.25 * (-0.643962658) * 0.634135591

* 0.643962658 * (1-0.643962658) = -
0.023406638
Value for changing weight 2: 0.25 * (-0.643962658) * 0.457602059
* 0.643962658 * (1-0.643962658) = -
0.016890593
Change weight 1: 0.35 + (-0.023406638) = 0.326593362
Change weight 2: 0.81 + (-0.016890593) = 0.793109407

Now we will change the weights in weight matrix 1:

Value for changing weight 1: 0.25 * (-0.643962658) * 0

* 0.634135591 * (1-0.634135591) = 0
Value for changing weight 2: 0.25 * (-0.643962658) * 0
* 0.457602059 * (1-0.457602059) = 0
Value for changing weight 3: 0.25 * (-0.643962658) * 1
* 0.634135591 * (1-0.634135591) = -
0.037351064
Value for changing weight 4: 0.25 * (-0.643962658) * 1
* 0.457602059 * (1-0.457602059) = -
0.039958271
Change weight 1: 0.62 + 0 = 0.62 (not changed)
Change weight 2: 0.42 + 0 = 0.42 (not changed)
Change weight 3: 0.55 + (-0.037351064) = 0.512648936
Change weight 4: -0.17+ (-0.039958271) = -0.209958271

The first input pattern had been propagated through the net.
The same procedure is used for the next input pattern, but then with the changed weight values.
After the forward and backward propagation of the second pattern, one learning step is complete and the net error can be
calculated by adding up the squared output errors of each pattern.
By performing this procedure repeatedly, this error value gets smaller and smaller.

The algorithm is successfully finished, if the net error is zero (perfect) or approximately zero.

Note that this algorithm is also applicable for Multi-Layer-Perceptrons with more than one hidden layer.

"What happens, if all values of an input pattern are zero?"

 Neural Networks with Java: neural net overview

If all values of an input pattern are zero, the weights in weight matrix 1 would never be changed for this pattern and the net
could not learn it. Due to that fact, a "pseudo input" is created, called Bias that has a constant output value of 1.

This changes the structure of the net in the following way:

These additional weights, leading to the neurons of the hidden layer and the output layer, have initial random values and are
changed in the same way as the other weights. By sending a constant output of 1 to following neurons, it is guaranteed that
the input values of those neurons are always differing from zero.

Selforganization

Selforganization is an unsupervised learning algorithm used by the Kohonen Feature Map neural net.

As mentioned in previous sections, a neural net tries to simulate the biological human brain, and selforganization is probably the
best way to realize this.
It is commonly known that the cortex of the human brain is subdivided in different regions, each responsible for certain functions.
The neural cells are organizing themselves in groups, according to incoming informations.
Those incoming informations are not only received by a single neural cell, but also influences other cells in its neighbourhood.
This organization results in some kind of a map, where neural cells with similar functions are arranged close together.

This selforganization process can also be performed by a neural network. Those neural nets are mostly used for classification
purposes, because similar input values are represented in certain areas of the net's map.
A sample structure of a Kohonen Feature Map that uses the selforganization algorithm is shown below:

http://rfhs8012.fh-regensburg.de/~saj39122/jfroehl/diplom/e-13-text.html (6 of 10) [1/1/2002 12:45:29 PM]

Neural Networks with Java: neural net overview

Kohonen Feature Map with 2-dimensional input and 2-dimensional map (3x3 neurons)

As you can see, each neuron of the input layer is connected to each neuron on the map. The resulting weight matrix is used to
propagate the net's input values to the map neurons.
Additionally, all neurons on the map are connected among themselves. These connections are used to influence neurons in a
certain area of activation around the neuron with the greatest activation, received from the input layer's output.

The amount of feedback between the map neurons is usually calculated using the Gauss function:

-|xc-xi|2
-------- where xc is the position of the most activated
neuron
2 * sig2 xi are the positions of the other map neurons
feedbackci = e sig is the activation area (radius)

In the beginning, the activation area is large and so is the feedback between the map neurons. This results in an activation of
neurons in a wide area around the most activated neuron.
As the learning progresses, the activation area is constantly decreased and only neurons closer to the activation center are
influenced by the most activated neuron.

Unlike the biological model, the map neurons don't change their positions on the map. The "arranging" is simulated by changing
the values in the weight matrix (the same way as other neural nets do).
Because selforganization is an unsupervised learning algorithm, no input/target patterns exist. The input values passed to the net's
input layer are taken out of a specified value range and represent the "data" that should be organized.

The algorithm works as follows:

1. Define the range of the input values

2. Set all weights to random values taken out of the input value range
3. Define the initial activation area
4. Take a random input value and pass it to the input layer neuron(s)
5. Determine the most activated neuron on the map:
Multiply the input layer's output with the weight values
The map neuron with the greatest resulting value is said to be "most activated"
Compute the feedback value of each other map neuron using the Gauss function
9. Change the weight values using the formula:
weight(old) + feedback value * ( input value - weight(old) ) * learning rate
Neural Networks with Java: neural net overview

11. Decrease the activation area

12. Go to step 4
13. The algorithm ends, if the activation area is smaller than a specified value

Example: see sample applet

The shown Kohonen Feature Map has three neurons in its input layer that represent the values of the x-, y- and z-dimension. The
feature map is initially 2-dimensional and has 9x9 neurons. The resulting weight matrix has 3 * 9 * 9 = 243 weights, because each
input neuron is connected to each map neuron.

In the beginning, when the weights have random values, the feature map is just an unordered mess.
After 200 learning cycles, the map has "unfolded" and a grid can be seen.
Neural Networks with Java: neural net overview

As the learning progresses, the map becomes more and more structured.
It can be seen that the map neurons are trying to get closer to their nearest blue input value.
Neural Networks with Java: neural net overview

At the end of the learning process, the feature map is spanned over all input values.
The reason why the grid is not very beautiful is that the neurons in the middle of the feature map are also trying to get closer to the
input values. This leads to a distorted look of the grid.
The selforganization is finished at this point.

I recommend you, to do your own experiments with the sample applet, in order to understand its behaviour. (A description of the
applet's controls is given on the belonging page)
By changing the net's parameters, it is possible to produce situations, where the feature map is unable to organize itself correctly.
Try, for example, to give the initial activation area a very small value or enter too many input values.

Types of neural nets top of page

navigation
[main page] [content] [neural net overview]
[class structure] [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you think ?]

Last modified: November 09, 1999

Copyright © 1996-97 Jochen Fröhlich. All rights reserved.
Neural Networks with Java: glossary

a
activation area refers to: Kohonen Feature Map
A value that indicates the area of influence of the
most activated neuron (the center of activation) on
other map neurons. The activation is spread out
around this center (maximum activation) and
decreases the greater the distance to this center is.
activation function refers to: neuron
A mathematical function that a neuron uses to
produce an output refering to its input value. Usually
this input value has to exceed a specified threshold
value that determines, if an output to other neurons
should be generated.
see also: hard limiter, signum activation, sigmoid
activation

API refers to: general term, abbreviation

Application Programming Interface.
All classes and routines that a programming
language implements. The API of Java is called
JDK.

artificial intelligence (AI) refers to: general term

A research discipline whose aim is to make
computers able to simulate human abilities,
especially the ability to learn. AI is separated in e.g.
neural net theory, expert systems, robotics, fuzzy
control systems, game theory etc.
AWT refers to: general term, abbreviation
Abstract Window Toolkit.
Classes and routines, used to create a user interface.
Java provides classes that are system-independent in
its package java.awt.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (1 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

axon refers to: neural cell

The part of a biological neural cell that contains the
dendrites, connecting this neural cell to other cells.
The incoming stimulation of a neural cell is
transported from the cell's core through the axon to
the outgoing connections.

b
backpropagation refers to: learning algorithm
A learning algorithm used by neural nets with
supervised learning. Special form of the delta
learning rule.
see also: forwardpropagation

Backpropagation Net refers to: neural net

A feedforward type neural net. Has one input layer,
one output layer and at least one hidden layer.
Mainly used for pattern association.
see also: backpropagation

bias refers to: neural net

A "pseudo" input of a neural net with any value
except zero. Its purpose is to generate different
inputs for different input patterns given to the net.

d
delta learning rule refers to: learning algorithm
A learning algorithm used by neural nets with
supervised learning. Effects the changing of weights
by multiplying a neuron's input with the difference
of its output and the desired output and the net's
learning rate.

dendrite refers to: neural cell

The connections between biological neural cells.
Electrical stimulation is transported from cell to cell
using these connections.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (2 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

e
error refers to: neural net, output
A value that indicates the "quality" of a neural net's
learning process. Used by neural nets with
supervised learning, by comparing the current output
values with the desired output values of the net. The
smaller the net's error is, the better the net had been
trained. Usually the error is always a value greater
than zero.

f
feedback type refers to: neural net
A specific connection structure of a neural net,
where neurons of one neuron layer may have
connections to neurons of other layers and also to
neurons of the same layer. An example of such a net
type is the Hopfield Net.
see also: feedforward type

feedforward type refers to: neural net

A specific connection structure of a neural net,
where neurons of one neuron layer may only have
connections to neurons of other layers. An example
of such a net type is the Perceptron.
see also: feedback type

forwardpropagation refers to: learning algorithm

The output values of a neural net's neurons are only
propagated through the net in one direction, from the
input layer to the output layer.
see also: backpropagation

g
GUI refers to: general term, abbreviation
Graphical User Interface.
The graphical environment of a software
application.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (3 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

h
hard limiter refers to: neuron, activation function
A specific type of a neuron's activation function.
see also: activation function

Hebb learning rule refers to: learning algorithm

A learning algorithm used by neural nets with
supervised learning, especially the Perceptron.
Effects the changing of weights by multiplying a
neuron's input with its output and the net's learning
rate.

hidden layer refers to: neural net

A type of neuron layer that lies between a neural
net's input and output layers. Called "hidden",
because its neuron values are not visible outside the
net. The usage of hidden layers extends a neural
net's abilities to learn logical operations.
see also: Multi-Layer-Perceptron

Hopfield Net refers to: neural net

A feedback type neural net. Has only one "layer",
build of a number of neurons, where each neuron is
connected to all other neurons. Mainly used for
pattern association.
see also: simulated annealing

i
input refers to: neuron, input layer
A set of values, called "pattern", that is passed to a
neural net's input neuron layer. The elements of
those patterns are usually binary values.
see also: input layer, output

input layer refers to: neuron layer, neural net

The first layer of a neural net, that accepts certain
input patterns and generates output values to the
successing weight matrix.
see also: input, weight matrix

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (4 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

IP refers to: general term, abbreviation

Internet Protocol.
A transmission protocol used to transport data across
the Internet.

j
JDK refers to: general term, abbreviation
Java Developers Kit.
An extensive set of Java classes, suitable for
different purposes. The classes are enclosed in
"packages", each covering a certain topic
(networking, user interface,...).
see also: API

k
Kohonen Feature Map refers to: neural net
A feedforward / feedback type neural net. Built of an
input layer thats neurons are connected with each
neuron of another layer, called "feature map". The
feature map can be one- or two-dimensional and
each of its neurons is connected to all other neurons
on the map. Mainly used for classification.
see also: selforganization

l
learning algorithm refers to: neural net
A mathematical algorithm that a neural net uses to
learn specific problems.
see also: backpropagation, delta learning rule,
forwardpropagation, Hebb learning rule, simulated
annealing

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (5 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

learning rate refers to: learning algorithm

A changeable value used by several learning
algorithms, which effects the changing of weight
values. The greater the learning rate, the more the
weight values are changed. Is usually decreased
during the learning process.

m
Multi-Layer-Perceptron refers to: neural net
A feedforward type neural net. Built of an input
layer, at least one hidden layer and one output layer.
Mainly used for pattern association.
see also: backpropagation, hidden layer, Perceptron

n
neuron refers to: neuron layer, neural net
An element of a neural net's neuron layer.
see also: activation function, threshold

neuron layer refers to: neural net

A layer of a neural net. The different layers of a
neural net are connected by weight matrices.
see also: input layer, hidden layer, output layer,
weight matrix

o
object orientation (OO) refers to: general term
A method of software-engineering. The main goal of
object orientation is to develope reusable software
components.
output refers to: neuron, output layer
A value or a set of values (pattern), generated by the
neurons of a neural net's output layer. Used to
calculate the current error value of the net.
see also: output layer, input, error, supervised
learning

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (6 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

output error see error

output layer refers to: neuron layer, neural net

The last layer of a neural net, that produces the
output value of the net.
see also: output

p
Perceptron refers to: neural net
A feedforward type neural net. Built of one input
layer and one output layer. Mainly used for pattern
association.
see also: Multi-Layer-Perceptron

propagation refers to: neural net, learning algorithm

The passing of values and errors through the
different layers of a neural net during its learning
process.
see also: forwardpropagation, backpropagation

propagation function refers to: neural net, learning algorithm

A function that is used to transport values through
the neurons of a neural net's layers. Usually, the
input values are added up and passed to an
activation function, which generates an output.
see also: forwardpropagation, backpropagation,
activation function

s
selforganization refers to: Kohonen Feature Map, learning algorithm
A learning algorithm used by the Kohonen Feature
Map neural net. During its learning process, the
neurons on the net's feature map are organizing
themselves depending on given input values. This
will result in a clustered neuron structure, where
neurons with similar properties (values) are arranged
in related areas on the map.
see also: Kohonen Feature Map, learning algorithm

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (7 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

sigmoid activation refers to: neuron, activation function

A specific type of a neuron's activation function.
see also: activation function

signum activation refers to: neuron, activation function

A specific type of a neuron's activation function.
see also: activation function

simulated annealing refers to: learning algorithm

A specific type of a learning algorithm, especially
for feedback type neural nets.
see also: Hopfield Net

supervised learning refers to: neural net, learning algorithm

A specific type of a learning algorithm. The output
(pattern) of the net is compared with a target output
(pattern). Depending on the difference between
these patterns, the net error is computed.
see also: unsupervised learning, error

t
TCP refers to: general term, abbreviation
Transmission Control Protocol.
A protocol used to transport data across the Internet.
thermodynamical model refers to: neural net
Another expression for feedback type neural nets.
Called "thermodynamical", because the term energy
is used instead of error.
see also: Hopfield Net

threshold refers to: neuron, activation function

A specific value that must be exceeded by a neuron's
activation function, before this neuron generates an
output.
see also: neuron, activation function

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (8 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

UDP refers to: general term, abbreviation

User Datagram Protocol.
A protocol used to transport data across the Internet
in form of datagrams (small packets that are sent
independently).
unsupervised learning refers to: neural net, learning algorithm
A specific type of a learning algorithm, especially
for selforganizing neural nets like the Kohonen
Feature Map. Unlike supervised learning, no target
patterns exist.
see also: supervised learning, selforganization

URL refers to: general term, abbreviation

Uniform Resource Locator.
An address of a certain location on the World Wide
Web. An URL usually has the following form:
http://www.[servername].[country|
category].

w
weight refers to: neural net, weight matrix
An element of a weight matrix. A connection
between two neurons with a value that is
dynamically changed during a neural net's learning
process.
see also: weight matrix, learning algorithm

weight matrix refers to: neural net

The connection structure between two neuron layers
of a neural net. Its elements, the weights, are
changed during the net's learning process. Each
neural net has at least one weight matrix.
see also: neuron layer, weight

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (9 of 10)15/7/2006 2:42:16 π•

Neural Networks with Java: glossary

xor problem refers to: general term

A logical operation with the following values:

x y XOR
false false false
false true true
true false true
true true false

Due to its contradiction, this function can only be

learned by neural nets with at least one hidden layer.
see also: Multi-Layer-Perceptron, hidden layer

navigation
[main page] [content] [neural net overview]
[class structure] [using the classes] [sample applet]
· glossary · [literature] [about the author] [what do you
think?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-g.html (10 of 10)15/7/2006 2:42:16 π•

 Neural Networks with Java

Diploma
of
Jochen Fröhlich

Fachhochschule Regensburg
Department of Computer Science

Supervised by Prof. Jürgen Sauer

copyright © 1996-97 Subject:

jochen fröhlich
all rights reserved
Neural Net Components
in an Object Oriented Class Structure

The classes are all written in Java (JDK Version 1.0.2)

and contain components of the Backpropagation Net
and the Kohonen Feature Map neural network types.

AWARDS

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-index.html (1 of 2)15/7/2006 3:42:36 π•

Neural Networks with Java

Considered Cool Rated Top 1% Web Applet

by Gamelan by JARS

"Thank You, I'm honored."

Web site sponsored by

navigation
· main page · [content] [neural net overview]
[class structure] [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you
think?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-index.html (2 of 2)15/7/2006 3:42:36 π•

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-author.html

name Jochen Fröhlich

studies Computer Science at the Fachhochschule

Regensburg

web site www.jochen-froehlich.de

I wish to thank

Beate Ott for checking the English language on these pages

...and for being a good friend

Prof. Jürgen Sauer for his lessons on neural networks (where it all
began) and his support while I did my diploma
(and to say "Just do it!" was definitely the right
way, I think...)

Prof. Dr. Fritz Jobst for his Java lessons.

Without them I probably would still write C
programs...

Daniela Fodor for explaining the secrets of object orientation

Fred Corbett for looking over my pages and checking my

English

James Gosling for his great idea

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-author.html (1 of 2)15/7/2006 3:43:16 π•

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-author.html

Grady Booch for his very good book 'Object Oriented Analysis
and Design'

Mary Campione, for their very helpful Java tutorial

Kathy Walrath

Gamelan and JARS for the cool fan and the delicious apple

Pink Floyd, for making very good and inspiring music

Jason Rebello

Esther Schweins for just looking great

all other Web authors for their ideas I use in my pages

Supporting

National Organization
Men Against Amazonian
Free speech online Rinderwahn online
Masterhood

navigation
[main page] [content] [neural net overview]
[class structure] [using the classes] [sample applet]
[glossary] [literature] · about the author · [what do you
think ?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-author.html (2 of 2)15/7/2006 3:43:16 π•

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-2.html

choose a section

What is Object Orientation?

Classes
Objects
Methods
Encapsulation
Inheritance
Abstraction

The classes and their

relationships

Neural net classes

Class NeuralNet
Class BackpropagationNet
Class KohonenFeatureMap

Common components
Class Neuron
Class NeuronLayer
Class WeightMatrix

Net type specific components

Class Pattern
Class MapNeuron
Class NeuronMatrix
Class InputValue
Class InputMatrix

navigation
[main page] [content] [neural net overview]
· class structure · [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you
think ?]

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-2.html (1 of 2)15/7/2006 3:44:52 π•

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-2.html

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-2.html (2 of 2)15/7/2006 3:44:52 π•

 Neural Networks with Java: class structure

The classes and their relationships bottom of page Common components

jump directly to

Neural net classes

Class NeuralNet
Class BackpropagationNet
Class
KohonenFeatureMap

Neural net classes

These classes represent two neural net types (Backpropagation Net and Kohonen Feature Map).
There is also an abstract class called "NeuralNet", which is the superclass of both types. This
class contains generic methods, e.g. a method to set the initial learning rate of a neural net.

Class NeuralNet

This class is the first class in the structure.

abstract class NeuralNet

extends java.lang.Object

instantiated can't be instantiated

by
constructors none

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (1 of 7)15/7/2006 3:45:05 π•

 Neural Networks with Java: class structure

methods boolean displayNow ()

Indicates, whether the net should be drawn or not,
depending on its display step. True, if the net should be
drawn. False otherwise.

boolean finishedLearning ()
Indicates that the net has finished learning. True, if the
learning process is finished. False otherwise.

String getElapsedTime ()
Returns the elapsed learning time of a neural net.

int getLearningCycle ()
Returns the current learning cycle of a neural net.

double getLearningRate ()
Returns the current learning rate of a neural net.

int getMaxLearningCycles ()
Returns the number of maximum learning cycles of a neural
net.

void resetTime ()
Resets the net's learning time.

void setDisplayStep ( int displayStep )

Sets a value that indicates the interval to display the net.

void setLearningRate ( double

learningRate )
Sets the learning rate of a neural net.

void setMaxLearningCycles ( int

maxLearningCycles )
Sets the number of learning cycles, the net shall perform.
If -1, the net has no maximum cycle.

top of page
Class BackpropagationNet

This class represents a Backpropagation Net neural net.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (2 of 7)15/7/2006 3:45:05 π•

Neural Networks with Java: class structure

public class BackpropagationNet

extends java.lang.Object
NeuralNet

instantiated application
by
constructors public BackpropagationNet ()
methods void addNeuronLayer ( int size )
Adds a neuron layer with size neurons.
Note that neuron layers are sequentially added to the net.

void connectLayers ()
Connects all neuron layers with weight matrices.
Must be called after all neuron layers have been added.

double getAccuracy ()
Returns the accuracy value.

double getError ()
Returns the current error of the net.

String getInputPattern ( int patternNr )

Returns the input pattern with number patternNr.

double getMinimumError ()
Returns the minimum error of a neural net.

float[] getNeuronOutputs ( int layerNr )

Returns the output values of all neurons in layer layerNr.

int getNumberOfLayers ()
Returns the number of neuron layers.

int getNumberOfNeurons ( int layerNr )

Returns the number of neurons in layer layerNr.

int getNumberOfPatterns ()
Returns the number of patterns.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (3 of 7)15/7/2006 3:45:05 π•

Neural Networks with Java: class structure

int getNumberOfWeights ()
Returns the number of weights of all weight matrices.

int getNumberOfWeights ( int matrixNr )

Returns the number of weights in weight matrix
matrixNr.

String getOutputPattern ( int

patternNr )
Returns the output pattern with number patternNr.

float getPatternError ( int patternNr )

Returns the error of output pattern patternNr.

String getTargetPattern ( int

patternNr )
Returns the target pattern with number patternNr.

float[][] getWeightValues ( int

matrixNr )
Returns the weight values of weight matrix matrixNr.
The values for matrixNr start with zero!

void learn ()
Performs one learning step.

synchronized void readConversionFile

( String conversionFileName )
Reads a conversion table for ASCII-binary values from file
conversionFileName.

synchronized void readPatternFile

( String patternFileName )
Reads input and target patterns from file
patternFileName.

String recall ( String recallInput )

Tries to recall the correct output for a learned input pattern
recallInput.

void setAccuracy ( double accuracy )

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (4 of 7)15/7/2006 3:45:05 π•

 Neural Networks with Java: class structure

Sets an accuracy value for the net, which is something like

a "fuzzy border" for output/recall purposes (default is 0.2).

void setMinimumError ( float

minimumError )
Sets the minimum error of a neural net.

top of page
Class KohonenFeatureMap

This class represents a Kohonen Feature Map neural net.

public class KohonenFeatureMap

extends java.lang.Object
NeuralNet

instantiated application
by
constructors public KohonenFeatureMap ()
methods void connectLayers ( InputMatrix
inputMatrix )
Connects the feature map and the input layer (which is
generated depending on the size of the inputMatrix)
with a weight matrix.

void createMapLayer ( int xSize, int

ySize )
Creates a two-dimensional feature map with
xSize*ySize map neurons.

double getActivationArea ()
Returns the current activation area.

double getInitActivationArea ()
Returns the initial activation area.

double getInitLearningRate ()
Returns the initial learning rate.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (5 of 7)15/7/2006 3:45:05 π•

Neural Networks with Java: class structure

int getMapSizeX ()
Returns the number of neurons in the map layer's x-
dimension.

int getMapSizeY ()
Returns the number of neurons in the map layer's y-
dimension.

int getNumberOfWeights ()
Returns the number of weights in the weight matrix.

double getStopArea ()
Returns the final activation area.

float[][] getWeightValues ()
Returns the weight values of the net's weight matrix.

void learn ()
Performs a learning step.

void setInitActivationArea ( double

initActivationArea )
Sets the initial activation area.

void setStopArea ( double stopArea )

Sets the final activation area at which the net stops learning.

void setInitLearningRate ( double

initLearningRate )
Sets the initial learning rate.

The classes and their relationships top of page Common components

navigation

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (6 of 7)15/7/2006 3:45:05 π•

Neural Networks with Java: class structure

[main page] [content] [neural net overview]

[class structure] [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you
think ?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-23-text.html (7 of 7)15/7/2006 3:45:05 π•

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-3.html

choose a section

Java = new ProgrammingLanguage()

Working with Java

The look of a class
Creating an object
Invoking a method

Using the classes in your own programs

Using the BackpropagationNet class
The structure of a conversion file
The structure of a pattern file
Using the KohonenFeatureMap
class
Using the InputMatrix class

Download

navigation
[main page] [content] [neural net overview]
[class structure] · using the classes · [sample applet]
[glossary] [literature] [about the author] [what do you
think ?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-3.html15/7/2006 3:46:42 π•
Neural Networks with Java: using the classes

Working with Java bottom of page Download

jump directly to

Using the classes in your own programs

Using the BackpropagationNet class
The structure of a conversion file
The structure of a pattern file
Using the KohonenFeatureMap
class
Using the InputMatrix class

Using the classes in your own programs

This section explains, how you can use the neural network classes in your own programs.

While designing the classes, I asked myself, what would be the easiest way to build a neural net?
I remembered the neural net lessons, where my professor used to say things like:

"Now we want to build a 3-layered Backpropagation Net with 4 neurons in its input layer
and 3 neurons in its output layer. The hidden layer consists of 2 neurons. Each neuron of
one layer is connected to all neurons of the following layer..."

While he said this, he drew a sketch of the net on the blackboard in the following way:

1. He drew the neurons of the input layer

2. He drew the neurons of the hidden layer
3. He drew the neurons of the output layer
4. He connected the input neurons with the hidden neurons
5. He connected the hidden neurons with the output neurons
6. He assigned random values to the weights of both weight matrices
7. He wrote down the input and target patterns
8. He demonstrated a few learning cycles of the net

For this procedure was always the same, and only the net structure changed, it seemed to be a
possible solution for an implementation.
So I came to the conclusion, building a neural net in a program should be done the same way as
you would describe it with your own words.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (1 of 14)15/7/2006 3:46:53 π•

 Neural Networks with Java: using the classes

The whole class structure consists of 11 classes, as can be seen in class structure/The classes and
their relationships, but you actually need to know more about only three of them to get your
neural net running.
The classes you explicitely use in your programs are: BackpropagationNet, KohonenFeatureMap,
and InputMatrix.

Using the BackpropagationNet class

Features of the BackpropagationNet class

number of minimum: 2, maximum: N
neuron layers
number of minimum: 1, maximum: N
neurons in each
layer
weight matrices automatically created and initialized
The weights are connecting the neurons
of two consecutive layers
bias values automatically used
number of input/ minimum: 1, maximum: N
target patterns
one learning step forward propagation and
includes backpropagation of all input patterns
(learning-by-epoch), changing of the
weights

Steps to create a Backpropagation Net

step sample code description
1. object BackpropagationNet bpn; REQUIRED
declaration Declares the object bpn to be
of type BackpropagationNet.
2. constructor bpn = new REQUIRED
call BackpropagationNet(); Creates an instance of
BackpropagationNet. This is
the only constructor of the
class and takes no arguments.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (2 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

3. read bpn.readConversionFile REQUIRED

conversion file ("fileName"); Reads the ASCII-binary
conversion file.
See The structure of a
conversion file below for
further explanation of
conversion files.
4. create input bpn.addNeuronLayer(i); REQUIRED
layer Creates the net's input layer
with i neurons.
The neuron layers are
sequentially added to the net
structure!
create hidden bpn.addNeuronLayer(h); optional
layer(s) Creates the net's hidden layer
(s) with h neurons. This step is
optional (if your net should
have no hidden layers) or may
be done one or more times (if
your net has one or more
hidden layers). If your net has
more than one hidden layers,
the number of neurons in each
layer must not always be the
same.
5. create output bpn.addNeuronLayer(o); REQUIRED
layer Creates the net's output layer
with o neurons.
6. connect all bpn.connectLayers(); REQUIRED
layers Connects the neurons of all
consecutive layers. All weight
matrices are created and
initialized with random values.
7. read pattern bpn.readPatternFile REQUIRED
file ("fileName"); Reads the pattern file that
contains the input and target
patterns.
See The structure of a
pattern file below for further
explanation of pattern files.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (3 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

set learning rate bpn.setLearningRate(x); optional

Sets the net's learning rate to x.
set minimum bpn.setMinimumError(x); optional
error Sets the net's minimum error to
x. The net learns, until its error
is smaller than this value.
set accuracy bpn.setAccuracy(x); optional
Sets the net's accuracy value to
x.
set maximum bpn.setMaxLearningCycles optional
learning cycles (x); Sets the maximum number of
learning cycles to x. The
default value is -1 (no
maximum).
reset learning bpn.resetTime(); optional
time Resets the learning time.
8. perform a bpn.learn(); REQUIRED
learning cycle Performs one learning cycle.
This method is usually called
within a loop, which exits, if
the finishedLearning()
method returns true.
recall a pattern bpn.recall optional
("inputPattern"); Tries to recall the correct
output of the input pattern
inputPattern.

information is available on:

accuracy bpn.getAccuracy(); optional
Returns the accuracy value of
the net.
elapsed time bpn.getElapsedTime(); optional
Returns the time that elapsed
since the learning process
started.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (4 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

input pattern bpn.getInputPattern(i); optional

Returns the input pattern with
number i.
Pattern numbers start with
zero!
learning cycle bpn.getLearningCycle(); optional
Returns the current learning
cycle of the net.
learning rate bpn.getLearningRate(); optional
Returns the learning rate of the
net.
minimum error bpn.getMinimumError(); optional
Returns the minimum error of
the net.
net error bpn.getError(); optional
Returns the current error of the
net.
neuron outputs bpn.getNeuronOutputs(i); optional
Returns the output values of all
neurons in layer i.
number of bpn.getNumberOfLayers(); optional
layers Returns the number of neuron
layers.
number of bpn.getNumberOfNeurons(i); optional
neurons Returns the number of neurons
in layer i.
Neuron layer numbers start
with zero!
number of bpn.getNumberOfPatterns(); optional
patterns Returns the number of input/
target patterns.
number of all bpn.getNumberOfWeights(); optional
weights Returns the number of all
weights.
number of bpn.getNumberOfWeights(m); optional
weights Returns the number of weights
in weight matrix m.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (5 of 14)15/7/2006 3:46:53 π•

 Neural Networks with Java: using the classes

output pattern bpn.getOutputPattern(i); optional

Returns the output pattern with
number i.
Pattern numbers start with
zero!
pattern error bpn.getPatternError(i); optional
Returns the error of the output
pattern with number i.
Pattern numbers start with
zero!
target pattern bpn.getTargetPattern(i); optional
Returns the target pattern with
number i.
Pattern numbers start with
zero!
weight values bpn.getWeightValues(m); optional
Returns all weight values of
weight matrix m.
Weight matrix numbers start
with zero!

Here is the source code of the BPN Application to show you an example: BPN.java

top of page
The structure of a conversion file

The conversion file of a Backpropagation Net must be used to convert ASCII characters (the net
input) to an internal binary representation.
The number of binary values that represent the ASCII characters can be changed freely, but has
to be the same for each character.

The general structure of a conversion file is as follows:

in the first line: number of conversions

each following line: at first position: the ASCII character

and then: the binary representation
(the length of this representation
doesn't matter,
but must be the same for each conversion)

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (6 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

Below you see the conversion file "ascii2bin.cnv" that is used in the BPN application.
It contains 64 conversions (as can be seen in the first line) and each ASCII character is converted
to 6 binary digits.

64
0000000
1000001
2000010
3000011
4000100
5000101
6000110
7000111
8001000
9001001
a001010
b001011
c001100
d001101
e001110
f001111
g010000
h010001
i010010
j010011
k010100
l010101
m010110
n010111
o011000
p011001
q011010
r011011
s011100
t011101
u011110
v011111
w100000
x100001
y100010
z100011
A100100

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (7 of 14)15/7/2006 3:46:53 π•

 Neural Networks with Java: using the classes

B100101
C100110
D100111
E101000
F101001
G101010
H101011
I101100
J101101
K101110
L101111
M110000
N110001
O110010
P110011
Q110100
R110101
S110110
T110111
U111000
V111001
W111010
X111011
Y111100
Z111101
?111110
?111111

Note: A conversion file must be read before you add neuron layers to the net, because the
number of binary digits must already be available when a neuron layer is created.

top of page
The structure of a pattern file

The pattern file of a Backpropagation Net contains the input and target patterns for the net.
The patterns should contain ASCII characters that are defined in the net's conversion file.
The number of patterns can be changed freely.
The length of an input pattern must be the same as the number of neurons in the net's input
neuron layer.
The length of a target pattern must be the same as the number of neurons in the net's output
neuron layer.

The general structure of a pattern file is as follows:

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (8 of 14)15/7/2006 3:46:53 π•

 Neural Networks with Java: using the classes

in the first line: number of patterns

in the second line: length of input patterns (same as number
of neurons in input layer)
in the third line: length of target patterns (same as
number of neurons in output layer)

each following line: the input pattern [whitespace] the

target pattern

Below you see the pattern file "towns.pat" that is used in the BPN application.
It contains 15 patterns (first line). Each input pattern consists of 10 ASCII characters (second
line) and each target pattern consists of 7 characters (third line).

15
10
7
Bonn000000 Germany
Brasilia00 Brasil0
Brussels00 Belgium
Helsinki00 Finland
London0000 England
Madrid0000 Spain00
Moscow0000 Russia0
New0Delhi0 India00
Oslo000000 Norway0
Paris00000 France0
Rome000000 Italy00
Stockholm0 Sweden0
Tokyo00000 Japan00
Vienna0000 Austria
Washington USA0000

Note: A pattern file must be read after the connectLayers() had been called.

top of page
Using the KohonenFeatureMap class

Features of the KohonenFeatureMap class

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (9 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

number of 1 input layer, 1 feature map

neuron layers
input dimension minimum: 1, maximum: 3
number of minimum: 1*1, maximum: N*M
neurons in
feature map
number of input minimum: 1, maximum: N
values
weight matrices automatically created and initialized
One matrix connects input layer and
feature map. The other matrix is not of
the WeightMatrix type and
connects the neurons of the feature
map among themselves.
biases not used
one learning step selection of a random input value,
includes finding the most activated map neuron,
changing of the weights

Steps to create a Kohonen Feature Map

step sample code description
1. object KohonenFeatureMap kfm; REQUIRED
declaration Declares the object kfm to be
of type KohonenFeatureMap.
2. constructor kfm = new KohonenFeatureMap REQUIRED
call (); Creates an instance of
KohonenFeatureMap. This is
the only constructor of the
class and takes no arguments.
3. create kfm.createMapLayer(xSize, REQUIRED
feature map ySize); Creates the net's feature map
with xSize*ySize map neurons.
4. define input see Using the InputMatrix class REQUIRED
matrix Creates the input matrix.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (10 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

5. connect kfm.connectLayers(im); REQUIRED

input layer Connects each input neuron
with feature (automatically created,
map depending on the dimension
of the input matrix im) with
each neuron of the feature
map. Besides, all map neurons
are connected among
themselves. All weight
matrices are created and
initialized with random
values, taken from the input
matrix im (created in step 4).
set initial kfm.setInitLearningRate(x); optional
learning rate Sets the net's initial learning
rate to x. The default value is
0.6.
set initial kfm.setInitActivationArea optional
activation area (x); Sets the net's initial activation
area to x. The default value is
the greater of both map sizes
divided by 2.
set final kfm.setStopArea(x); optional
activation area Sets the net's final activation
area to x. The default value is
initActivationArea divided by
10.
set maximum kfm.setMaxLearningCycles optional
learning cycles (x); Sets the maximum number of
learning cycles to x. The
default value is -1 (no
maximum).
reset learning kfm.resetTime(); optional
time Resets the learning time.
6. perform a kfm.learn(); REQUIRED
learning cycle Performs one learning cycle.
This method is usually called
within a loop, which exits, if
the finishedLearning()
method returns true.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (11 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

information is available on:

activation area kfm.getActivationArea(); optional
Returns the current activation
area of the feature map.
elapsed time kfm.getElapsedTime(); optional
Returns the time that elapsed
since the learning process
started.
final activation kfm.getStopArea(); optional
area Returns the final activation
area of the feature map.
initial kfm.getInitActivationArea optional
activation area (); Returns the initial activation
area of the feature map.
initial learning kfm.getInitLearningRate(); optional
rate Returns the initial learning
rate of the net.
learning cycle kfm.getLearningCycle(); optional
Returns the current learning
cycle of the net.
learning rate kfm.getLearningRate(); optional
Returns the current learning
rate of the net.
map size in x kfm.getMapSizeX(); optional
Returns the size of the feature
map in x-dimension.
map size in y kfm.getMapSizeY(); optional
Returns the size of the feature
map in y-dimension.
number of kfm.getNumberOfWeights(); optional
weights Returns the number of weights
in the weight matrix that
connects the input neurons
with the neurons of the feature
map.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (12 of 14)15/7/2006 3:46:53 π•

 Neural Networks with Java: using the classes

weight values kfm.getWeightValues(); optional

Returns all weight values of
the weight matrix that
connects the input neurons
with the neurons of the feature
map.

top of page
Using the InputMatrix class

Features of the InputMatrix class

number of input minimum: 1, maximum: N
values
dimensions minimum: 1, maximum: 3

Steps to create an Input Matrix

step sample code description
1. object InputMatrix im; REQUIRED
declaration Declares the object im to be of
type InputMatrix.
2. constructor call im = new InputMatrix REQUIRED
(size,dim); Creates an instance of
InputMatrix. This is the only
constructor of the class and has
the following arguments:
size: the number of input values
(1...N)
dim: the dimension of this matrix
(1...3)

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (13 of 14)15/7/2006 3:46:53 π•

Neural Networks with Java: using the classes

3. set values im.setInputX(x); (and) REQUIRED

(version 1) im.setInputY(y); (and) Sets input values for each
im.setInputZ(z); dimension to this matrix.
These methods can be used
sequentially, if you want an input
matrix with 1, 2, or 3
dimensions. The arguments x, y,
and z are arrays of type int.
To define a 3-dimensional input
matrix, you may use the
setInputValues(x,y,z)
method.
3. set values im.setInputValues(x,y, REQUIRED
(version 2) z); Sets input values for all three
dimensions to this matrix.
This method is used, when you
want an input matrix with three
dimensions. The arguments x, y,
and z are arrays of type int.

information is available on:

number of input im.size(); optional
values Returns the number of input
values in this input matrix.
dimension im.getDimension(); optional
Returns the dimension of this
input matrix.

Working with Java top of page Download

navigation
[main page] [content] [neural net overview]
[class structure] [using the classes] [sample applet]
[glossary] [literature] [about the author] [what do you
think ?]

Copyright 1996-97 Jochen Fröhlich. All rights reserved. A new version is available.

http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-33-text.html (14 of 14)15/7/2006 3:46:53 π•