Deep Learning

[email protected]
KP2WFEJ3RX

S U N I L KU M A R V U P PA L A
S U N I L .V U P PA L A @ G M A I L .C O M
W W W. L I N K E D I N .C O M / I N / S U N I LV U P PA L A /

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Agenda
Session-1:
• DL - What, Where, Why, How? • Building DL models using Keras+Tensorflow
• Why deep learning now? • Convolutional Neural Network
• Applications of DL
• Machine learning vs Deep learning
[email protected]
KP2WFEJ3RX

• Fundamentals of Artificial neural network

• Tensorflow playground
•Session-2:
• Feed forward networks
• Various layers in DL
• Activation Functions
• Hyper parameters in DL
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 2
 What is Deep learning (DL)?
Deep learning (DL) is a class of machine learning (ML)
algorithms that*:
• use a cascade of many layers of nonlinear processing units
for feature extraction and transformation
• Each successive layer uses the output from the previous
[email protected]
KP2WFEJ3RX
layer as input
• learn multiple levels of representations that correspond to
different levels of abstraction

DL is inspired by the structure and function of the brain called

artificial neural networks.
*wikipedia

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 3
 Where it fits in?

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 4
 Where to use DL?
• Finance domain (Categorical and Numerical data) :
• Identify the fraud detection in credit card transactions

•Healthcare domain (Image data):

• Lung cancer classification of images

•Social media(Image data):

[email protected]
• Face recognition and tag the people
KP2WFEJ3RX

•Across the domains (Text):

• Identify the potential cases of automation from historical ticket data using
• Build a chat bot

Few more applications of DL are: Personalized recommendations, Prediction, Anomaly detection, Drug
discovery, Autonomous cars, Video analytics etc...

•But is it NEW concept?

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 5
 Brief history

Convolution Neural Networks for Handwritten Recognition Google Brain Project on 16k
1958 Perceptron 1974 Backpropagation 1998 Cores
[email protected] 2012
KP2WFEJ3RX

awkward silence (AI Winter)

1969 1995 2006 2012

Perceptron criticized SVM reigns Restricted AlexNet wins
Boltzmann ImageNet
Machine

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 6
 Why second wave?
•More data from systems and sensors (IoT)

•More compute power : GPUs, multi-core CPUs

[email protected]
KP2WFEJ3RX

Important property:

Results get better with more data + bigger models +

more computation

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 7
 Machine Learning vs Deep Learning

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 8
 Supervised & unsupervised learning (Recap)

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 9
 Artificial neural network

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 10
 Tensorflow platground demo
A neural net model is composed a set of Layers.
Run multiple examples in the increasing order of
complexity
◦ Linear
◦ Comlicated circle
[email protected]
KP2WFEJ3RX
◦ Spiral
◦ Shallow learning
◦ Deep learning

There are many types of layers available and each

layer has many parameters. Thus we can have
infinitely many different network architectures.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 11
[email protected]

Deep Learning Fundamentals

KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Artificial neural network
Artificial neurons are elementary units in an
artificial neural network.

The artificial neuron receives one or more inputs

KP2WFEJ3RX (representing dendrites) and sums them to produce
[email protected]

an output (or activation) (representing a neuron's

axon).

Usually the sums of each node are weighted, and

the sum is passed through a non-linear function
known as an activation function.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 13
 Back propagation
With the concept of gradient descent
◦ Forward propagation: Activation [input * weight
matrix]
◦ To get optimal value of each weight:
◦ Direction is opp. to the gradient
[email protected]
KP2WFEJ3RX ◦ Find weight with minimum error
◦ Derivative (slope of a tangent line - rate of change of a
function)
◦ Partial derivative (wrt one of the variables)
◦ Chain rule (derivatives of composite functions)
◦ calculate the error wrt each weight
◦ New weight = old weight - Derivative Rate * learning
rate

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 14
 Feed forward nets
Information flow is unidirectional
• Data is presented to Input layer
• Passed on to Hidden Layer
[email protected]
KP2WFEJ3RX
• Passed on to Output layer
 Information is distributed
 Information processing is parallel
 Backpropagation

• Requires training set (input / output pairs)

• Starts with small random weights
• Error is used to adjust weights (supervised learning)

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 15
 Solving XOR with a neural net

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 16
 Basic set of Layers
Dense Layer
Dropout Layer
Convolution1D
[email protected]
KP2WFEJ3RX

Convolution2D
MaxPooling1D
LSTM

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 17
 Dense and Dropout layers
Dense Layer:
It creates a regular fully connected Neural net layer
Dense (output_dim , activation='linear')
◦ output_dim: (integer > 0 ) Specifies the size of the Layer ( Number of Neurons)
◦ activation: name of activation function
[email protected]
KP2WFEJ3RX

Dropout Layer:
Dropout: A Simple Way to Prevent Neural Networks from Over- fitting
Dropout ( p )
Applies Dropout to the input. Dropout consists in randomly setting a fraction p of the input units
to 0 at each update during the training phase, which helps prevent over-fitting.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 18
 Convolution1D/2D
Convolution operator for filtering neighborhoods of one-dimensional inputs.
1D convolution layer (e.g. temporal convolution)
2D convolution layer (e.g. spatial convolution over images)
Convolution1D ( nb_filter, filter_length, activation='linear', border_mode='valid',
[email protected]
KP2WFEJ3RX
subsample_length=1 )
◦ nb_filter: Number of convolution kernels to use (dimensionality of the output).
◦ filter_length: The extension (spatial or temporal) of each filter.
◦ activation: name of activation function to use
◦ border_mode: 'valid' or 'same'.
◦ subsample_length: factor by which to subsample output.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 19
 MaxPooling1D
Max pooling operation for temporal data.
The max-pooling layer would reduce the input Matrix into a down sampled size with max value for each block.
Please refer to the image shown below for an example.

[email protected]
KP2WFEJ3RX

MaxPooling1D (pool_length=2, stride=None, border_mode='valid' )

pool_length: size of the region to which max pooling is applied
stride: integer, or None. factor by which to downscale. 2 will halve the input. If None, it will default to
pool_length.
border_mode: 'valid' or 'same'

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 20
 Activation Functions
In Neural Networks, the activation function of a node defines the output of that node given an
input or set of inputs.

A standard computer chip circuit can be seen as a digital network of activation functions that
can be "ON" (1) or "OFF" (0), depending on input.
[email protected]
KP2WFEJ3RX

This is similar to the behavior of the linear perceptron in neural networks.

It is the nonlinear activation function that allows such networks to compute nontrivial problems
using only a small number of nodes.
ReLu: max(0,x)

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 21
 Properties of activation function
◦ Nonlinear:
◦ When the activation function is non-linear, then a two-layer neural network can be proven to be a universal function approximator.
◦ Continuously differentiable:
◦ This property is necessary for enabling gradient-based optimization methods.
◦ Range:
[email protected]◦ When the range of the activation function is finite, gradient-based training methods tend to be more stable.
KP2WFEJ3RX
◦ Smaller learning rates are typically necessary.
◦ Monotonic:
◦ When the activation function is monotonic, the error surface associated with a single-layer model is guaranteed to be convex.
◦ Smooth
◦ Functions with a Monotonic derivative have been shown to generalize better in some cases.
Approximates identity near the origin:
◦ The neural network will learn efficiently when its weights are initialized with small random values.
◦ When the activation function does not approximate identity near the origin, special care must be used when initializing the
weights.
◦ https://en.wikipedia.org/wiki/Activation_function

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 22
 Activation functions
2
20

Logistic
2
18 1.5

16

14
1.5

1
Hyperbolic tangent 1

1
0.5
12
0.5

y=
0
10

exp( x) − exp(− x)
8 0 -0.5

y= 1 + exp(− x)
6 -0.5 -1

exp( x) + exp(− x)
-1 -1.5
2
-1.5 -2
0 -10 -8 -6 -4 -2 0 2 4 6 8 10
0 2 4 6 8 10 12 14 16 18 20
-2
[email protected] -10 -8 -6 -4 -2 0 2 4 6 8 10

KP2WFEJ3RX
Linear
y=x
Rectifier / ramp function
f(x) = max(0,x)
x is the input to a neuron.
smooth approximation to the rectifier
is softplus
f(x) = ln(1+ex)

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 23
 Activation functions
Unit step (threshold):
The transfer function translates the input signals to
output signals. Four types of transfer functions are
commonly used, Unit step (threshold), sigmoid,
piecewise linear, and Gaussian.
[email protected]
KP2WFEJ3RX

Sigmoid:
The sigmoid function consists of 2 functions, logistic
and tangential. The values of logistic function range
from 0 and 1 and -1 to +1 for tangential function.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 24
 Deep Learning Algorithms
MLP – Multi Layer perceptron
◦ A multilayer perceptron (MLP) is a feed forward artificial neural network model that maps sets of input
data onto a set of appropriate outputs.
◦ An MLP consists of multiple layers of nodes in a directed graph, with each layer fully connected to the
next one.
◦ Had multiple hidden layers with logistic regression classifier transformation
[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 25
 Parameters to vary for tuning
•Number of layers •Batch size
•Number of neurons in each layer •Momentum parameter (weightage given to
earlier steps taken in the process of gradient
•Activation function in each layer
descent)
•Number of epochs
[email protected]
KP2WFEJ3RX •Kernels
•Error/loss functions
•Number of features
•Iteration (equivalent to when a weight update is
done) •Number of filters for images

•Learning rate (α) •Filter sizes for images

• Size of the step in the direction of the •Gradient descent methods
negative gradient

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 26
 Recap of evaluation measures
Accuracy: Overall, how often is the classifier correct?
(TP+TN)/total = (100+50)/165 = 0.91

Misclassification Rate: Overall, how often is it wrong?

(FP+FN)/total = (10+5)/165 = 0.09
[email protected]
KP2WFEJ3RX
equivalent to 1 minus Accuracy also known as "Error Rate"

False Positive Rate: When it's actually no, how often does it predict
yes?
Precision: When it predicts yes, how often FP/actual no = 10/60 = 0.17
is it correct?
TP/predicted yes = 100/110 = 0.91 Specificity: When it's actually no, how often does it predict no?
TN/actual no = 50/60 = 0.83
"Sensitivity" or "Recall": When it's actually equivalent to 1 minus False Positive Rate
yes, how often does it predict yes?
TP/actual yes = 100/105 = 0.95
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 27
 Keras and Demos
Why we need Keras?
◦ Keras: Deep Learning library for Theano and
TensorFlow
◦ An API spec for building DL models across many
platforms
[email protected]

Guiding principles: modularity, minimalism,

KP2WFEJ3RX

extensibility, and Python-nativeness Other alternate frameworks:

Simple • Caffe
• Tensorflow
Keras’ community is growing, while Theano’s is
• Torch/PyTorch
declining
Less flexible
Less projects available online than caffe
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 28
 Keras+Tensorflow Demos

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 29
[email protected]

Convolutional Neural Networks

KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Convolutional Neural Network
CNN - Convolutional Neural Network
◦ Feed-forward artificial neural network
◦ Convolutional networks were inspired by
biological processes
[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 31
 Convolution operation

[email protected]
KP2WFEJ3RX

• Non-linearity is needed to learn complex (non-linear) representations of data, otherwise the NN

would be just a linear function

• Most deep networks use ReLU - max(0,x), since it trains much faster, is more expressive than logistic
function and prevents the gradient vanishing problem.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 32
 Convolution operation
Raw Image Pixel Filter or Kernel or Feature detector

[email protected]
KP2WFEJ3RX

Convolved Feature or
Activation Map or the
Feature Map.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 33
 Convolutional Neural Network
An image input constitutes a 3-dimensional structure called the Input Volume (255x255x3).

CNN’s use filters as kernels where the parameters or weights have to be learnt.A filter is a matrix of lower
size than the input to it.

The inputs are convolved with the filters and passed through the activation function.
[email protected]
KP2WFEJ3RX

The weights of the kernels are randomly initialized and are modified during training based on error-
minimization using backpropagation.

The real values of the kernel matrix change with each learning iteration over the training set, indicating that
the network is learning to identify which regions are of significance for extracting features from the data.

Stride: The shift of filter after each convolution.It can be increased from 1 to a larger value to decrease
overfitting.
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 34
 ReLu and Max pooling

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 35
 CNN in summary

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 36
[email protected]

RNN and LSTM

KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Recurrent Networks
Feed forward networks:
◦ Information only flows one way
◦ One input pattern produces one output
◦ No sense of time (or memory of previous state)
Recurrency
[email protected]
KP2WFEJ3RX
◦ Nodes connect back to other nodes or themselves
◦ Information flow is multidirectional
◦ Sense of time and memory of previous state(s)

Possible applications of RNN’s are in domains where data is sequential.

For example:
Speech and Text (NLP)
Music
Protein and DNA sequences
Time series from trade data
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 38
 RNN and LSTM

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 39
 LSTM (Long Short Term Memory)
It create a layer of Long-Short Term Memory units .
LSTM (output_dim , activation='tanh', inner_activation='hard_sigmoid')
output_dim: dimension of the internal projections and the final output.
activation: name of activation function to use
[email protected]
KP2WFEJ3RX

Inner_activation: name of activation function to use for inner cells

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 40
 Long short-term memory
LSTM - Long short-term memory
◦ Recurrent neural network (RNN)
◦ Take input not just the current input
example they see, but also what they
perceived one step back in time.
Feedback loop, ingesting their own
[email protected]
KP2WFEJ3RX

outputs moment after moment as input

◦ an LSTM network is well-suited to learn
from experience to classify, process and
predict time series
◦ LSTM blocks contain three or four "gates" that
they use to control the flow of information into
or out of their memory.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 41
 Text analysis

[email protected]
KP2WFEJ3RX

http://slideplayer.com/slide/9885340/
Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Type of chatbots
Usecases:
• Pizza Hut to help you order a pizza
• Uber to book a taxi
• CNN to keep you up-to-date with news
[email protected]
KP2WFEJ3RX content

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 43
 Conversational interfaces

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 44
 Pretrained models
[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 45
 Convnets and pretrained models
• LeNet (1990s)

• AlexNet (2012)

• ZF Net (2013)
[email protected]
KP2WFEJ3RX

• GoogLeNet (2014)

• VGGNet (2014)

• ResNets (2015)

• DenseNet (August 2016)

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 46
 Architecture of Alex Krizhevsky et al.

● 8 layers total

● Trained on Imagenet Dataset (1000

[email protected]
KP2WFEJ3RX
categories, 1.2M training images,
150k test images)

● 16.4% top-5 error

○ Winner of the ILSVRC- 2012
challenge.

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Impact on Computer Vision
ImageNet Challenge2012

1.2M images with 1000 object categories

[email protected]
KP2WFEJ3RX

• AlexNet of uni Toronto: 15% error rate vs 26% for

2th placed (traditional CV)

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 Practical tips
[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
 DL Project ideas
Image Speech
◦ Captioning ◦ Alexa / Home APIs
◦ extract embedded text ◦ Local languages
◦ emoji - extract sentiment
[email protected] Numerical / Categorical
KP2WFEJ3RX
◦ Volume, result prediction
Text
◦ Time series forecasting - weather / server
◦ Sarcasm
◦ Satillite data analysis - ISRO
◦ Chatbots - specific topic
◦ Govt data analysis
◦ Sentiment analysis
◦ http://data.gov.in
◦ Local languages

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 50
 DL project ideas contd
Video Robot
◦ analytics - speed control ◦ Path planning / recommendations
◦ search ◦ reinforcement learning

◦ annotated data

Multimodal
[email protected]
KP2WFEJ3RX
Recommender systems
◦ Chatbots ◦ Specific product / item category
◦ Get information from multiple sources
◦ Generative

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 51
 Practical challenges and tips while using
ML / DL
◦ Data availability
◦ Rules based + ML

◦ Application revisited: Gien ticket data of client, identify the potential candidates for
[email protected]
KP2WFEJ3RX
◦ Used ML for clustering the ticket data with preprocessing
◦ Tried ML algorithms including ensemble to reach
◦ Extended DL algorithms but the improvement is not more than 5%

◦ Discussion

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 52
 Cheat sheets

[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 53
 Learning path
Building knowledge : Practise with assignments and projects:
K1. Refresh your fundamentals on statistics, P1. Kaggle challenges with available data
probability and linear algebra Cuisine prediction, lung cancer,
K2. Do Course era deep learning Choose text or image or
categorical/nurmerical probems
K3. Refer specific concepts in
[email protected]
KP2WFEJ3RX

deeplearningbook.org by Goodfellow P2. Participate in hackathons and assesments

K4. Refer advanced topics of deep learning P3. Github profile and upload your codes
based on the need - Generative adversial P4. Define your problem with your domain
networks, Auto encoders, deep reinforcement experience and follow steps of data science
learning, visualization techniques project execution with github repositories
K5. Attend webinars and AV meets or
conferences to network and see latest trends

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 54
 Learning path references
K1. Relearning K4. Advanced topics

https://in.udacity.com/course/intro-to-descriptive-statistics-- https://www.youtube.com/watch?v=RtxI449ZjSc&feature=
ud827 relmfu

https://www.khanacademy.org/math/linear-algebra
P1. Competitions
K2. Online courses:
[email protected]
KP2WFEJ3RX
https://www.kaggle.com/competitions
https://www.coursera.org/specializations/deep-learning
http://www.image-net.org/challenges/LSVRC/
https://in.udacity.com/course/deep-learning--ud730
P2. Assessement of your skills
K3: Books:
https://datahack.analyticsvidhya.com/contest/all/
http://www.deeplearningbook.org/
P3. Github profile
http://neuralnetworksanddeeplearning.com/
http://deeplearning.stanford.edu/tutorial/

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 55
 References
• http://machinelearningmastery.com/4-steps-to-get-started-in-machine-learning/
• https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
• https://archive.ics.uci.edu/ml/datasets/Adult
• http://www.datasciencecentral.com/profiles/blogs/the-data-science-project-lifecycle
• https://project.inria.fr/deeplearning/files/2016/05/DLFrameworks.pdf
• http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/
[email protected]
KP2WFEJ3RX • http://cs231n.stanford.edu/

• http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf
• http://www.kdnuggets.com/software/text.html
• http://neuralnetworksanddeeplearning.com/
• deepmind.com/blog
• http://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/
• https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxhcmlqaXRsYWhhfGd4OjdhNzhiOTd
lZjM4NTRiOWY
• Machine learning, Deep learning courses in CourseEra by Andrew NG
• https://becominghuman.ai/cheat-sheets-for-ai-neural-networks-machine-learning-deep-learning-big-data-678c51b4b463

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action. 56
 Thank you
[email protected]
KP2WFEJ3RX

Proprietary content. © Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.