Introduction to ANN
SCO 411
Dr. Elijah Maseno
SCO 411 1
Biological Neural Network
• Biological neural network describes about the working
principle of human brain.
• Brain is a most powerful computing machine from others.
• The inner working of human brain is built on concept of
neurons and the networks of the neurons known as biological
neural network.
• The brain contains more than 86 billion neurons. The neurons
are connected and communicate with other neurons through
axons.
• Dendrites are used for taking input from external
environment or sensory organs.
• The electrical signal created by the input and these are
quickly pass through the neural network and send to the
other neuron through synapse to handle the issue.
SCO 411 2
Artificial Neural Network (ANN)
• Neural network or ANN is based on biological neural
network.
• Here multiple nodes are act as neurons. The neurons or
nodes are interconnected and communicate with each other
by links.
• ANN contains three layers. First layer is known as input
layer, second layer is known as hidden layer and third layer
is known as output layer. Each layer contains one or more
neurons.
• The nodes of the input layer can take input data and
perform operations on it and send the results of these
operations to other neurons of the hidden layer. The hidden
layer sends data to the output layer.
• The output of each node is known as its activation or node
value.
• For increase the problem solving capabilities, we can
increase the number of hidden layers and number 4 of neurons
in any given layer, and number of paths between neurons.
5
• In the network or model, each link assigned with some
weight.
• Weight is nothing an integer number that controls the signal
between the two neurons. If the network generates a “good
or desired” output, there is no need to adjust the weights.
• However, if the network generates a “poor or undesired”
output or an error, then the system update the weights in
order to improve subsequent results.
Perceptron and Multi-Layer
Perceptron
Perceptron is a simple form of Neural Network and consists of
a single layer where all the mathematical computations are
performed.
7
Whereas, Multilayer Perceptron also known as Artificial
Neural Networks consists of more than one perceptron which
is grouped together to form a multiple layer neural network.
8
Now in the above picture, you can see each neuron’s detailed view.
Here, each neurons have some weights (in above picture w1, w2, w3)
and biases and based on this computations are done as, combination =
bias + weights * input (F = w1*x1 + w2*x2 + w3*x3) and finally
activation function is applied output = activation(combination) in
above picture activation is sigmoid represented by 1/(1 + e-F).
9
Working of ANN
• At First, information is feed into the input layer which then
transfers it to the hidden layers, and interconnection
between these two layers assign weights to each input
randomly at the initial point and then bias is added to each
input neuron and after this, the weighted sum which is a
combination of weights and bias is passed through the
activation function.
• Activation Function has the responsibility of which node to
fire for feature extraction and finally output is calculated.
This whole process is known as Foreword Propagation.
• After getting the output model to compare it with the
original output and the error is known and finally, weights
are updated in backward propagation/back propagation to
reduce the error and this process continues for a certain
number of epochs (iteration). Finally, model weights get
updated and prediction is done.
10
