How to Create a Machine Learning Model using Keras
CBIIT/FNL Workshop
Frederick National Laboratory for Cancer Research
Bioinformatics Analyst IV
Hyun Jung
November 6, 2019
1
Outline
• Why Deep Learning?
• Why Keras?
• How does it work?
• Convolution, ReLU, Max pooling, Fully-connected layer
• Gradient Descent and Backpropagation
• Know your data
• Build/Train/Test your own network
• Glossary
2
Why Deep Learning?
ICDAR 2011 Chinese handwriting IJCNN 2011 traffic signs
ISBI 2012 brain segmentation ICPR 2012 cancer detection MICCAI 2013 Grand Challenge
3
Why Deep Learning?
4
Why Keras?
From: Google Trends
5
Why Keras?
From: TensorFlow official website
6
How does it work?
If (circles):
if (brown-ish):
if (two triangles):
picture = dog
else:
picture = food
Traditional Programming Dog Cookie
Image Source: guidedogs.org Image Source: foodnetwork.com
7
How does it work?
Dog vs. Food
Image source: boredpanda.com
8
How does it work?
Dog Food
Food Dog
Dog Food
. Dog
.
.
Dog Food
Food Dog
Food
9
How does it work?
• Building Blocks of Convolutional Neural Network (CNN)
• Convolution, ReLU, Pooling, and Fully-Connected (Dense) Layer
Simple convolutional neural network architecture
Image Source: MathWorks
10
How does it work?
• Why the CNN structured in such a way?
Hierarchy of visual detectors of increasing complexity achieves sophisticated perceptual categorization,
with the higher levels being able to recognize 1000's of different objects, people, etc.
Image Source: Medicine Library
11
How does it work?
• Input
Matrix representation of a digit 8 Composition of RGB from 3 Grayscale images
Image credit: Deep Learning made easy with Deep Cognition from Medium Image credit: Wikipedia
12
How does it work?
• Convolution
Convolution operation
Image credit: the data science blog
Convolution operation and its effect
Image credit: Deep Learning Methods for Vision (CVPR 12 tutorial) Image credit: groundai.com
13
How does it work?
• Convolution
Convolution operation
Image credit: the data science blog
Convolution operation and its effect
Image credit: Deep Learning Methods for Vision (CVPR 12 tutorial) Image credit: An Intuitive Explanation of Convolutional Neural Networks
14
How does it work?
• ReLU (Rectified Linear Unit)
• Introducing non-linearity
ReLU operation and its effect
Image credit: the data science blog
15
How does it work?
• Pooling (sub-sampling or down-sampling)
• Reduce dimensionality and extract most import information
Max pooling operation Max pooling effect
http://cs231n.github.io/convolutional-networks/ Neural Networks, MLSS2015 Summer School, Facebook AI Research,
16
How does it work?
• Story so far
Simple convolutional neural network architecture
Learned features from a CNN
Image Source: MathWorks H. Lee, et al., PROC. OF THE 26th ICML, 2009
17
How does it work?
GoogLeNet structure and visualization of its feature maps
Image Source: Feature Visualization from Distill 18
How does it work?
• Fully Connected Layer
Simple convolutional neural network architecture
Image Source: MathWorks
19
How does it work?
• Fully Connected Layer
Image Source: Understanding Convolutional Neural Networks, Medium
20
How does it work?
• What are we training?
• Weights and biases
Image credit: Deep Learning made easy with Deep Cognition from Medium
21
How does it work?
• How to train?
• Gradient Descent and Backpropagation
Gradient descent mechanism Backpropagating error to individual weights
Image Source: Image Source:
Gradient Descent: All You Need to Know from Rohan & Lenny #1: Neural Networks & The
Medium Backpropagation Algorithm, Explained from Medium
22
How does it work?
• How to train?
• Gradient Descent is too slow
Different optimization methods on loss surface contours Optimization methods on saddle point
Image Source: Sebastian Ruder webpage Image Source: Sebastian Ruder webpage
23
Know Your Data
• Understanding your data is CRUCIAL
• Kaggle: Breast Histopathology Images
• URL: https://www.kaggle.com/paultimothymooney/breast-histopathology-images
• Invasive Ductal Carcinoma Breast Cancer Images
• Scanned at 40x optical magnification
• Image Size: 50 x 50 x 3 (Height x Width x Channels)
24
Know Your Data
• Understanding your data is CRUCIAL
• Kaggle: Breast Histopathology Images
• Total number of patients: 279 (Train: 223, Valid: 28, Test: 28)
• Total number of benign images: 198,783
• Total number of malignant images: 78,786
• Average number of benign images per patient: 712.32
• Average number of malignant images per patient: 282.38
• Maximum/Minimum number of benign images among patients: 2231/14
• Maximum/Minimum number of malignant images among patients: 1347/10
25
Build/Train/Test your own network
• Download/Analyze/Organize Data (Done, Organize_data.py)
• Setup a virtual environment (anaconda/mybinder) and install necessary packages
• Read/Explore data
• Create generators to feed images to your network
• Define your own network
• Compile and train your network
• Save and load trained model for testing
• Transfer learning (if time allows)
26
Glossary
• Batch size: Number of images that are trained simultaneously
• Step/Iteration: Training one batch
• One Epoch: Training entire image once
• Overfitting: Network is specifically optimized to solve training images
• Underfitting: Network performance is not fully utilized
• Dropout: Randomly discard information to avoid overfitting
Image Source: What is underfitting and overfitting in machine learning and how to deal with it., Medium
27
Glossary
• Loss/objective function: Binary cross entropy function
−(𝑦𝑦 * log(𝑝𝑝) + (1−𝑦𝑦) * log(1−𝑝𝑝)) − (y*log(𝑝𝑝)+(1−y)*log(1− 𝑝𝑝))
prediction/Label y=0 y=1
p=0 0
p=1 0
28
Environment Setup
• Anaconda (Your own laptop, can be faster)
• conda create -n MyEnvName python=3
• conda install pip
• pip install --upgrade tensorflow or tensorflow-gpu
• pip install Keras
• conda install nb_conda
• pip install scikit-learn
• pip install matplotlib
• pip install scikit-image
• Mybinder.org (Jupyter server)
• https://mybinder.org
• GitHub repository name or URL
• https://github.com/newhyun00/DL_Workshop
• Press launch
29
