Perceptron

Perceptron is a single layer neural network

• Perceptron is a linear classifier (binary).
• Also, it is used in supervised learning.
• It helps to classify the given input data
• A perceptron is a neural network unit (an artificial neuron) that does
certain computations to detect features or business intelligence in the
input data.
Rise of Artificial Neurons (Based on Biological
Neuron)
• Researchers Warren McCullock and Walter Pitts published their first
concept of simplified brain cell in 1943 called McCullock-Pitts (MCP)
neuron.
• They described such a nerve cell as a simple logic gate with binary
outputs.
 What is Artificial Neuron
• An artificial neuron is a mathematical function based on a model of
biological neurons, where each neuron takes inputs, weighs them
separately, sums them up and passes this sum through a nonlinear function
to produce output.
The artificial neuron characteristics:

• A neuron is a mathematical function modeled on the working of

biological neurons
• It is an elementary unit in an artificial neural network
• One or more inputs are separately weighted
• Inputs are summed and passed through a nonlinear function to
produce output
• Every neuron holds an internal state called activation signal
• Each connection link carries information about the input signal
• Every neuron is connected to another neuron via connection link
Perceptron
There are two types of Perceptrons: Single layer and Multilayer.
• Single layer - Single layer perceptrons can learn only linearly separable
patterns
• Multilayer - Multilayer perceptrons or feedforward neural networks
with two or more layers have the greater processing power
• The Perceptron algorithm learns the weights for the input signals in
order to draw a linear decision boundary.
• This enables distinguish between the two linearly separable classes
+1 and -1.
Perceptron is a function that maps its input “x,” which is multiplied with
the learned weight coefficient; an output value ”f(x)”is generated.
Inputs of a Perceptron
Activation Functions of Perceptron
Output of Perceptron
Output of Perceptron
 But how does it work?
The perceptron works on these simple steps
• a. All the inputs x are multiplied with their weights w. Let’s call it k.
• b. Add all the multiplied values and call them Weighted Sum.
Why do we need Weights and Bias?
Weights shows the strength of the particular node.
A bias value allows you to shift the activation function curve up or down.

Why do we need Activation

Function?
In short, the activation functions
are used to map the input
between the required values like
(0, 1) or (-1, 1).
 Error in Perceptron
• In the Perceptron Learning Rule, the predicted output is compared with
the known output. If it does not match, the error is propagated
backward to allow weight adjustment to happen.
 Perceptron: Decision Function
The decision function φ(z) of Perceptron is defined to take a linear combination
of x and w vectors.

The value z in the decision function is given by:

1 otherwise.
The decision function is +1 if z is greater than a threshold θ, and it is -

This is the Perceptron algorithm.

 Bias Unit
For simplicity, the threshold θ can be brought to the left and represented as
w0x0, where w0= -θ and x0= 1.

The value w0 is called the bias unit.

The decision function then becomes:

 Implement Logic Gates with Perceptron
Perceptron - Classifier Hyperplane
• The Perceptron learning rule converges if the two classes can be
separated by the linear hyperplane.
• However, if the classes cannot be separated perfectly by a linear
classifier, it could give rise to errors.
• The classifier boundary for a binary output in a Perceptron is
represented by the equation :
 What is Logic Gate?
• Logic gates are the building blocks of a digital system, especially neural
networks.
• They are the electronic circuits that help in addition, choice, negation,
and combination to form complex circuits.
• Using the logic gates, Neural Networks can learn on their own without
you having to manually code the logic.
• Most logic gates have two inputs and one output.
• Each terminal has one of the two binary conditions, low (0) or high (1),
represented by different voltage levels. The logic state of a terminal
changes based on how the circuit processes data.
• Based on this logic, logic gates can be categorized into seven types:
AND , NAND , OR, NOR, NOT, XOR, XNOR
XOR Gate with Neural Networks
• An XOR gate assigns weights so that XOR conditions are met. It cannot
be implemented with a single layer Perceptron and requires Multi-
layer Perceptron or MLP.
• H represents the hidden layer, which allows XOR implementation.
• I1, I2, H3, H4, O5are 0 (FALSE) or 1 (TRUE)
• t3= threshold for H3; t4= threshold for H4; t5= threshold for O5
• H3= sigmoid (I1*w13+ I2*w23–t3); H4= sigmoid (I1*w14+ I2*w24–t4)
• O5= sigmoid (H3*w35+ H4*w45–t5);
Sigmoid Activation Function
Sigmoid Curve
• This is called a logistic sigmoid
• probability of the value
between 0 and 1.
• Used when one is interested in
probability mapping rather
than precise values of input
parameter t.
• The sigmoid output is close to
zero for highly negative input
& can lead to slow learning