Scribd
Upload
65 views

Applied Machine Learning For Engineers: Artificial Neural Networks

1. The document discusses building and training a neural network model for handwritten digit recognition using the MNIST dataset in TensorFlow and Keras. 2. It loads the MNIST dataset, preprocesses the images, splits the data into train, validation and test sets, and builds a simple multi-layer perceptron model. 3. The model is trained for 20 epochs and its performance is evaluated on the validation and test sets, achieving over 96% accuracy on the test set. Learning curves and a sample prediction are also presented.

Uploaded by

Gilbe Testa
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
65 views

Applied Machine Learning For Engineers: Artificial Neural Networks

1. The document discusses building and training a neural network model for handwritten digit recognition using the MNIST dataset in TensorFlow and Keras. 2. It loads the MNIST dataset, preprocesses the images, splits the data into train, validation and test sets, and builds a simple multi-layer perceptron model. 3. The model is trained for 20 epochs and its performance is evaluated on the validation and test sets, achieving over 96% accuracy on the test set. Learning curves and a sample prediction are also presented.

Uploaded by

Gilbe Testa
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Applied Machine Learning for Engineers

FS 2020 - B. Vennemann

Artificial Neural Networks

In [1]:

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

np.random.seed(0)

Install Tensorflow
conda install tensorflow

Update Numpy
conda update numpy

OSX Error workaround


conda install nomkl

Check installation

In [2]:

import tensorflow as tf
from tensorflow import keras

print("Tensorflow version:", tf.__version__)


print("Keras version:", keras.__version__)

Tensorflow version: 2.0.0


Keras version: 2.2.4-tf

Load the MNIST dataset

In [3]:

mnist = keras.datasets.mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

print("Training set size :", X_train.shape)


print("Test set size :", X_test.shape)

Training set size : (60000, 28, 28)


Test set size : (10000, 28, 28)
Display the first 20 images

In [4]:

plt.figure(figsize=(18, 10))

for i in range(20):
plt.subplot(4, 5, i+1)
plt.imshow(X_train[i].reshape((28, 28)), cmap='gray')
plt.text(24, 3, str(y_train[i]), fontsize=12, color='white')
plt.xticks([])
plt.yticks([])
plt.show()

Preprocess data

In [5]:

print("Max:", X_train.max())
print("Min:", X_train.min())

Max: 255
Min: 0

In [6]:

X_train = X_train / 255.0


X_test = X_test / 255.0

Create validation set to monitor training

In [7]:

X_train, X_val = X_train[:50000], X_train[50000:]


y_train, y_val = y_train[:50000], y_train[50000:]

print("Training set size :", X_train.shape)


print("Validation set size :", X_val.shape)
print("Test set size :", X_test.shape)

Training set size : (50000, 28, 28)


Validation set size : (10000, 28, 28)
Test set size : (10000, 28, 28)
Create the ANN using the Keras Sequential API

In [8]:

model = keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape=[28, 28])) # turns (batch_size x 28 x 28) into
(batch_size x 784)
model.add(keras.layers.Dense(300, activation="relu")) # hidden layer
model.add(keras.layers.Dense(10, activation="softmax")) # output layer

or creating the ANN in one go

In [9]:

model = keras.models.Sequential([
keras.layers.Flatten(input_shape=[28, 28]),
keras.layers.Dense(300, activation="relu"),
keras.layers.Dense(10, activation="softmax")
])

Display the model

In [10]:

model.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_1 (Flatten) (None, 784) 0
_________________________________________________________________
dense_2 (Dense) (None, 300) 235500
_________________________________________________________________
dense_3 (Dense) (None, 10) 3010
=================================================================
Total params: 238,510
Trainable params: 238,510
Non-trainable params: 0
_________________________________________________________________

Compile the model

In [11]:

model.compile(loss="sparse_categorical_crossentropy",
optimizer="sgd",
metrics=["accuracy"])

Train the model


In [12]:

history = model.fit(X_train, y_train, epochs=20, batch_size=32,


validation_data=(X_val, y_val))

Train on 50000 samples, validate on 10000 samples


Epoch 1/20
50000/50000 [==============================] - 26s 514us/sample - loss: 0.667
8 - accuracy: 0.8361 - val_loss: 0.3628 - val_accuracy: 0.9034
Epoch 2/20
50000/50000 [==============================] - 20s 400us/sample - loss: 0.352
0 - accuracy: 0.9039 - val_loss: 0.2929 - val_accuracy: 0.9203
Epoch 3/20
50000/50000 [==============================] - 18s 351us/sample - loss: 0.301
3 - accuracy: 0.9161 - val_loss: 0.2614 - val_accuracy: 0.9288
Epoch 4/20
50000/50000 [==============================] - 16s 321us/sample - loss: 0.269
9 - accuracy: 0.9253 - val_loss: 0.2410 - val_accuracy: 0.9340
Epoch 5/20
50000/50000 [==============================] - 16s 328us/sample - loss: 0.246
6 - accuracy: 0.9314 - val_loss: 0.2224 - val_accuracy: 0.9398
Epoch 6/20
50000/50000 [==============================] - 18s 351us/sample - loss: 0.226
9 - accuracy: 0.9367 - val_loss: 0.2068 - val_accuracy: 0.9435
Epoch 7/20
50000/50000 [==============================] - 20s 400us/sample - loss: 0.210
7 - accuracy: 0.9415 - val_loss: 0.1923 - val_accuracy: 0.9469
Epoch 8/20
50000/50000 [==============================] - 19s 379us/sample - loss: 0.196
4 - accuracy: 0.9448 - val_loss: 0.1827 - val_accuracy: 0.9505
Epoch 9/20
50000/50000 [==============================] - 15s 301us/sample - loss: 0.184
0 - accuracy: 0.9487 - val_loss: 0.1732 - val_accuracy: 0.9540
Epoch 10/20
50000/50000 [==============================] - 16s 310us/sample - loss: 0.172
9 - accuracy: 0.9518 - val_loss: 0.1655 - val_accuracy: 0.9556
Epoch 11/20
50000/50000 [==============================] - 17s 343us/sample - loss: 0.163
3 - accuracy: 0.9542 - val_loss: 0.1568 - val_accuracy: 0.9579
Epoch 12/20
50000/50000 [==============================] - 20s 404us/sample - loss: 0.154
7 - accuracy: 0.9566 - val_loss: 0.1501 - val_accuracy: 0.9593
Epoch 13/20
50000/50000 [==============================] - 17s 350us/sample - loss: 0.146
5 - accuracy: 0.9588 - val_loss: 0.1456 - val_accuracy: 0.9604
Epoch 14/20
50000/50000 [==============================] - 17s 335us/sample - loss: 0.139
6 - accuracy: 0.9612 - val_loss: 0.1391 - val_accuracy: 0.9630
Epoch 15/20
50000/50000 [==============================] - 18s 361us/sample - loss: 0.133
0 - accuracy: 0.9632 - val_loss: 0.1354 - val_accuracy: 0.9645
Epoch 16/20
50000/50000 [==============================] - 19s 374us/sample - loss: 0.127
1 - accuracy: 0.9646 - val_loss: 0.1325 - val_accuracy: 0.9650
Epoch 17/20
50000/50000 [==============================] - 16s 328us/sample - loss: 0.121
5 - accuracy: 0.9661 - val_loss: 0.1274 - val_accuracy: 0.9660
Epoch 18/20
50000/50000 [==============================] - 11s 222us/sample - loss: 0.116
4 - accuracy: 0.9680 - val_loss: 0.1237 - val_accuracy: 0.9662
Epoch 19/20
50000/50000 [==============================] - 11s 215us/sample - loss: 0.111
7 - accuracy: 0.9695 - val_loss: 0.1195 - val_accuracy: 0.9672
Epoch 20/20
50000/50000 [==============================] - 11s 218us/sample - loss: 0.107
5 - accuracy: 0.9704 - val_loss: 0.1163 - val_accuracy: 0.9670

Plot the learning curves


In [13]:

loss = history.history['loss']
val_loss = history.history['val_loss']
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']

epochs = np.arange(1, len(loss)+1)


plt.figure(figsize=(16, 5))
plt.subplot(1,2,1)
plt.plot(epochs, loss, label='training loss')
plt.plot(epochs, val_loss, label='validation loss')
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend()
plt.grid(True)

plt.subplot(1,2,2)
plt.plot(epochs, acc, label='training accuracy')
plt.plot(epochs, val_acc, label='validation accuracy')
plt.xlabel('epoch')
plt.ylabel('accuracy')
plt.legend()
plt.grid(True)
plt.plot()
plt.show()

Plotting training history using Pandas


In [14]:

import pandas as pd

df = pd.DataFrame(history.history)
df.plot(figsize=(8, 5))
plt.grid(True)
plt.show()

Making a prediction using the trained model

In [16]:

i = 15
y_pred = model.predict(X_test[i].reshape(1, 28, 28))
print(y_pred)
print(y_pred.sum())

[[3.62362363e-04 7.51586573e-04 3.04447574e-04 2.38896813e-02


1.27459425e-04 9.70107079e-01 1.51503045e-04 1.00383026e-04
4.13918169e-03 6.63491373e-05]]
1.0

In [17]:

y_pred = model.predict_classes(X_test[i].reshape(1, 28, 28))


print("Predictied class : ", y_pred[0])
print("Actual class : ", y_test[i])

Predictied class : 5
Actual class : 5

Evaluate test set performance

In [18]:

test_loss, test_acc = model.evaluate(X_test, y_test, batch_size=64, verbose=0)

print("Test set loss : ", test_loss)


print("Test set acc. : ", test_acc)

Test set loss : 0.11765392680168152


Test set acc. : 0.9668

In [ ]:

You might also like