Artificial Intelligence and Machine Learning (Anna University)

II Year/IV Semester
CS3491 – Artificial Intelligence & Machine Learning

Laboratory Manual
 COURSE OBJECTIVES:

1. Study about uninformed and Heuristic search techniques.

2. Learn techniques for reasoning under uncertainty
3. Introduce Machine Learning and supervised learning algorithms
4. Study about ensembling and unsupervised learning algorithms
5. Learn the basics of deep learning using neural networks

EXPERIMENTS LIST
1. Implementation of Uninformed search algorithms (BFS, DFS)

2. Implementation of Informed search algorithms (A*, memory-bounded A*)

3. Implement naïve Bayes models

4. Implement Bayesian Networks

5. Build Regression models

6. Build decision trees and random forests

7. Build SVM models

8. Implement ensembling techniques

9. Implement clustering algorithms

10. Implement EM for Bayesian networks

11. Build simple NN models

12. Build deep learning NN models

 COURSE OUTCOMES:

On completion of the course, students will be able to:

CO1: Use appropriate search algorithms for problem solving

CO2: Apply reasoning under uncertainty

CO3: Build supervised learning models

CO4: Build ensembling and unsupervised models CO5:

Build deep learning neural network models

TEXT BOOKS:
1. Stuart Russell and Peter Norvig, “Artificial Intelligence – A Modern Approach”,
Fourth Edition, Pearson Education, 2021.
2. Ethem Alpaydin, “Introduction to Machine Learning”, MIT Press, Fourth
Edition, 2020.

REFERENCES:
1. Dan W. Patterson, “Introduction to Artificial Intelligence and Expert Systems”,
Pearson Education,2007
2. Kevin Night, Elaine Rich, and Nair B., “Artificial Intelligence”, McGraw Hill, 2008
3. Patrick H. Winston, "Artificial Intelligence", Third Edition, Pearson Education, 2006
4. Deepak Khemani, “Artificial Intelligence”, Tata McGraw Hill Education,
2013 (http://nptel.ac.in/)
5. Christopher M. Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006.
6. Tom Mitchell, “Machine Learning”, McGraw Hill, 3rd Edition,1997.
7. Charu C. Aggarwal, “Data Classification Algorithms and Applications”, CRC Press, 2014
8. Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar, “Foundations of
Machine Learning”, MIT Press, 2012.
9. Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”, MIT Press
 INDEX
1. Implementation of Uninformed search algorithms (BFS, DFS)

Aim:
To implement uninformed search algorithms such as BFS and DFS.

Algorithm:
Step 1:= Initialize an empty list called 'visited' to keep track of the nodes visited during the
traversal.
Step 2:= Initialize an empty queue called 'queue' to keep track of the nodes to be
traversed in the future.
Step 3:= Add the starting node to the 'visited' list and the 'queue'.
Step 4:= While the 'queue' is not empty, do the following:
a. Dequeue the first node from the 'queue' and store it in a variable called 'current'.
b. Print 'current'.
c. For each of the neighbours of 'current' that have not been visited yet, do
the following:
i. Mark the neighbour as visited and add it to the 'queue'.
Step 5:= When all the nodes reachable from the starting node have been visited,
terminate the algorithm.

Breadth First Search

: Program :
graph = {
'5' : ['3','7'],
'3' : ['2', '4'],
'7' : ['8'],
'2' : [],
'4' : ['8'],
'8' : []
}
visited =
[] queue =
[]
def bfs(visited, graph,
node):
visited.append(node)
queue.append(node)
while queue:
m = queue.pop(0)
print (m, end = " ")
for neighbour in graph[m]:
if neighbour not in visited:
visited.append(neighbour)
queue.append(neighbour)

print("Following is the Breadth-First Search")

bfs(visited, graph, '5')
Output:

Following is the Breadth-First

Search 5 3 7 2 4 8

Depth first Search:

Algorithm:
Step 1:= Initialize an empty set called 'visited' to keep track of the nodes visited during the
traversal.
Step 2:= Define a DFS function that takes the current node, the graph, and the 'visited'
set as input.
Step 3:= If the current node is not in the 'visited' set, do the following:
a. Print the current node.
b. Add the current node to the 'visited' set.
c. For each of the neighbours of the current node, call the DFS function recursively
with the neighbour as the current node.
Step 4:= When all the nodes reachable from the starting node have been visited,
terminate the algorithm.

graph = {
'5' : ['3','7'],
'3' : ['2', '4'],
'7' : ['8'],
'2' : [],
'4' : ['8'],
'8' : []
}

visited = set()
def dfs(visited, graph,
node): if node not in
visited:
print (node)
visited.add(node
)
for neighbour in graph[node]:
dfs(visited, graph,
neighbour)

print("Following is the Depth-First

Search") dfs(visited, graph, '5')
Output:

Following is the Depth-First

Search 5
3
2
4
8
7

Result:

Thus the uninformed search algorithms such as BFS and DFS have been
executed successfully and the output got verified
 2. Implementation of Informed search

algorithm (A*) Aim:

To implement the informed search algorithm A*.

Algorithm:

1. Initialize the distances dictionary with float('inf') for all vertices in the
graph except for the start vertex which is set to 0.
2. Initialize the parent dictionary with None for all vertices in the graph.
3. Initialize an empty set for visited vertices.
4. Initialize a priority queue (pq) with a tuple containing the sum of the
heuristic value and the distance from start to the current vertex, the distance
from start to the current vertex, and the current vertex.
5. While pq is not empty, do the following:
a. Dequeue the vertex with the smallest f-distance (sum of the heuristic
value and the distance from start to the current vertex).
b. If the current vertex is the destination vertex, return distances and parent.
c. If the current vertex has not been visited, add it to the visited set.
d. For each neighbor of the current vertex, do the following:
i. Calculate the distance from start to the neighbor (g) as the sum of the
distance from start to the current vertex and the edge weight between the
current vertex and the neighbor.
ii. Calculate the f-distance (f = g + h) for the neighbor.
iii. If the f-distance for the neighbor is less than its current distance in
the distances dictionary, update the distances dictionary with the new
distance and the parent dictionary with the current vertex as the parent of
the neighbor.
iv. Enqueue the neighbor with its f-distance, distance from start to
neighbor, and the neighbor itself into the priority queue.
6. Return distances and parent.

Program :

import heapq

def a_star(graph, start, dest, heuristic):

distances = {vertex: float('inf') for vertex in graph} distances[start] = 0

parent = {vertex: None for vertex in graph}

visited = set()

pq = [( 0 + heuristic[start], 0 , start)] # E space

while pq:
curr_f, curr_dist, curr_vert = heapq.heappop(pq)

if curr_vert not in visited:

visited.add(curr_vert)
 for nbor,weight in graph[curr_vert].items():
distance = curr_dist + weight # distance from start (g)
f_distance = distance + heuristic[nbor] # f = g + h

# Only process new vert if it's f_distance is

lower if f_distance < distances[nbor]:
distances[nbor] = f_distance
parent[nbor] = curr_vert

if nbor == dest:
# we found a path based on heuristic
return distances, parent

heapq.heappush(pq, (f_distance, distance, nbor)) #logE time

return distances, parent

def generate_path_from_parents(parent, start, dest):
path = []
curr =
dest while
curr:
path.append(curr)
curr =
parent[curr]

return '->'.join(path[::-1])

graph = {
'A': {'B':5, 'C':5},
'B': {'A':5, 'C':4, 'D':3 },
'C': {'A':5, 'B':4, 'D':7, 'E':7, 'H':8},
'D': {'B':3, 'C':7, 'H':11, 'K':16, 'L':13,
'M':14}, 'E': {'C':7, 'F':4, 'H':5},
'F': {'E':4, 'G':9},
'G': {'F':9, 'N':12},
'H': {'E':5, 'C':8, 'D':11, 'I':3 },
'I': {'H':3, 'J':4},
'J': {'I':4, 'N':3},
'K': {'D':16, 'L':5, 'P':4, 'N':7},
'L': {'D':13, 'M':9, 'O':4, 'K':5},
'M': {'D':14, 'L':9, 'O':5},
'N': {'G':12, 'J':3, 'P':7},
'O': {'M':5, 'L':4},
'P': {'K':4, 'J':8, 'N':7},
}
heuristic =
{ 'A': 16,
 'B': 17,
'C': 13,
'D': 16,
'E': 16,
'F': 20,
'G': 17,
'H': 11,
'I': 10,
'J': 8,
'K': 4,
'L': 7,
'M': 10,
'N': 7,
'O': 5,
'P': 0
}

start = 'A'
dest= 'P'
distances,parent = a_star(graph, start, dest, heuristic)
print('distances => ', distances)
print('parent => ', parent)
print('optimal path => ', generate_path_from_parents(parent,start,dest))

Output:

distances => {'A': 0, 'B': 22, 'C': 18, 'D': 24, 'E': 28, 'F': 36, 'G': inf, 'H': 24, 'I': 26, 'J': 28, 'K': 28,
'L': 28, 'M': 32, 'N': 30, 'O': 30, 'P': 28}
parent => {'A': None, 'B': 'A', 'C': 'A', 'D': 'B', 'E': 'C', 'F': 'E', 'G': None, 'H': 'C', 'I': 'H', 'J': 'I',
'K': 'D', 'L': 'D', 'M': 'D', 'N': 'J', 'O': 'L', 'P': 'K'}
optimal path => A->B->D->K->P

Result:
Thus the program to implement informed search algorithm have been
executed successfully and output got verified.
3. Implement Naïve Bayes models.

Aim:

To diagnose the climate dataset with Naïve Bayes Classifier Algorithm.

Algorithm:

1. Import the required libraries: numpy, matplotlib.pyplot, pandas, seaborn.

2. Load the dataset from the given CSV file "NaiveBayes.csv" using the
pandas "read_csv()" function.
3. Separate the input and output variables from the dataset by using
"iloc" and "values" methods and assign them to "X" and "y"
variables respectively.
4. Split the dataset into training and testing datasets using the
"train_test_split()" function from the "sklearn.model_selection"
module. Assign the split data to "X_train", "X_test", "y_train" and
"y_test" variables.
5. Standardize the input data using the "StandardScaler()" function
from the "sklearn.preprocessing" module. Scale the training data
and testing data separately and assign them to "X_train" and
"X_test" variables.
6. Create a Bernoulli Naive Bayes classifier object using the
"BernoulliNB()" function from the "sklearn.naive_bayes" module and
assign it to the "classifer" variable.
7. Train the Bernoulli Naive Bayes classifier using the "fit()" method of
the "classifer" object by passing the "X_train" and "y_train" variables as
arguments.
8. Predict the output values for the test dataset using the "predict()"
method of the "classifer" object and assign them to "y_pred" variable.
9. Calculate the accuracy score of the model by passing the predicted
output values "y_pred" and actual output values "y_test" to the
"accuracy_score()" function from the "sklearn.metrics" module and
print the result.
10. Create a Gaussian Naive Bayes classifier object using the "GaussianNB()"
function from the "sklearn.naive_bayes" module and assign it to the
"classifer1" variable.
11. Train the Gaussian Naive Bayes classifier using the "fit()" method of the
"classifer1" object by passing the "X_train" and "y_train" variables as
arguments.
12. Predict the output values for the test dataset using the "predict()"
method of the "classifer1" object and assign them to "y_pred1"
variable.
13. Calculate the accuracy score of the model by passing the predicted
output values "y_pred1" and actual output values "y_test" to the
"accuracy_score()" function from the "sklearn.metrics" module and
print the result.
Program:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
dataset =
pd.read_csv('NaiveBayes.csv') #
split the data into inputs and
outputs X = dataset.iloc[:,
[0,1]].values
y = dataset.iloc[:, 2].values
from sklearn.model_selection import
train_test_split # assign test data size 25%
X_train, X_test, y_train, y_test =train_test_split(X,y,test_size= 0.25,
random_state=0) from sklearn.preprocessing import StandardScaler
# scalling the
input data sc_X =
StandardScaler()
X_train =
sc_X.fit_transform(X_train)
X_test =
sc_X.fit_transform(X_test)
from sklearn.naive_bayes import
BernoulliNB # initializaing the NB
classifer =
BernoulliNB() #
training the
model
classifer.fit(X_train,
y_train) # testing the
model
y_pred = classifer.predict(X_test)
from sklearn.metrics import
accuracy_score # printing the
accuracy of the model
print(accuracy_score(y_pred,
y_test))
from sklearn.naive_bayes import
GaussianNB # create a Gaussian
Classifier
classifer1 =
GaussianNB() #
training the model
classifer1.fit(X_train, y_train)

# testing the model

y_pred1 = classifer1.predict(X_test)
from sklearn.metrics import
accuracy_score # printing the
accuracy of the model
print(accuracy_score(y_test,y_pred1)
)

Result Thus the program with Naïve Bayes Classifier Algorithm have been executed
successfully and output got verified
 4. Implement Bayesian Networks

Aim:
To construct a Bayesian network, to demonstrate the diagnosis of heart
patients using standard Heart Disease Data Set.

Algorithm:

Step 1: Import required modules

Step 2: Define network structure
Step 3: Define the parameters using CPT
Step 4: Associate the parameters with the model
structure Step 5: Check if the cpds are valid for the
model
Step 6: View nodes and edges of the model
Step 7: Check independencies of a node
Step 8: List all Independencies

Program:

from pgmpy.models import BayesianNetwork

from pgmpy.inference import
#
VariableElimination
Defining network structure

alarm_model =
BayesianNetwork (
[
("Burglary", "Alarm"),
("Earthquake", "Alarm"),
("Alarm", "JohnCalls"),
("Alarm", "MaryCalls"),
]
)

# Defining the parameters using CPT

from pgmpy.factors.discrete import TabularCPD

cpd_burglary = TabularCPD(
variable="Burglary", variable_card=2, values=[[0.999], [0.001]]
)
cpd_earthquake = TabularCPD(
variable="Earthquake", variable_card=2, values=[[0.998], [0.002]]
)
cpd_alarm = TabularCPD(
variable="Alarm",
variable_card=2,
values=[[0.999, 0.71, 0.06, 0.05], [0.001, 0.29, 0.94, 0.95]],
evidence=["Burglary", "Earthquake"],
evidence_card=[2, 2],
)
cpd_johncalls =
TabularCPD(
variable="JohnCalls",
 values=[[0.95, 0.1], [0.05, 0.9]],
evidence=["Alarm"],
evidence_card=[2],
)
cpd_marycalls = TabularCPD(
variable="MaryCalls",
variable_card=2,
values=[[0.1, 0.7], [0.9, 0.3]],
evidence=["Alarm"],
evidence_card=[2],
)

# Associating the parameters with the model structure

alarm_model.add_cpds(
cpd_burglary, cpd_earthquake, cpd_alarm, cpd_johncalls, cpd_marycalls
)
# Checking if the cpds are valid for the model
alarm_model.check_model()

# Viewing nodes of the model

alarm_model.nodes()

# Viewing edges of the model

alarm_model.edges()

# Checking independcies of a node

alarm_model.local_independencies("Burglary")

# Listing all Independencies

alarm_model.get_independencies()

Output:

True
NodeView(('Burglary', 'Alarm', 'Earthquake', 'JohnCalls', 'MaryCalls'))
OutEdgeView([('Burglary', 'Alarm'), ('Alarm', 'JohnCalls'), ('Alarm', 'MaryCalls'),
('Earthquake', 'Alarm')])
(Burglary ⟂ Earthquake)
(MaryCalls ⟂ Earthquake, Burglary, JohnCalls | Alarm) (MaryCalls ⟂
Burglary, JohnCalls | Earthquake, Alarm)
(MaryCalls ⟂ Earthquake, JohnCalls | Burglary,
Alarm) (MaryCalls ⟂ Earthquake, Burglary |
JohnCalls, Alarm) (MaryCalls ⟂ JohnCalls |
Earthquake, Burglary, Alarm) (MaryCalls ⟂ Burglary |
Earthquake, JohnCalls, Alarm) (MaryCalls ⟂
Earthquake | Burglary, JohnCalls, Alarm) (JohnCalls ⟂
Earthquake, Burglary, MaryCalls | Alarm) (JohnCalls
⟂ Burglary, MaryCalls | Earthquake, Alarm)
(JohnCalls ⟂ Earthquake, MaryCalls | Burglary,
Alarm) (JohnCalls ⟂ Earthquake, Burglary |
MaryCalls, Alarm) (JohnCalls ⟂ MaryCalls |
Earthquake, Burglary, Alarm)
(JohnCalls ⟂ Burglary | Earthquake, MaryCalls,
Alarm) (JohnCalls ⟂ Earthquake | Burglary,
MaryCalls, Alarm) (Earthquake ⟂ Burglary)
(Earthquake ⟂ MaryCalls, JohnCalls | Alarm)
(Earthquake ⟂ MaryCalls, JohnCalls | Burglary, Alarm)
(Earthquake ⟂ JohnCalls | MaryCalls, Alarm)
(Earthquake ⟂ MaryCalls | JohnCalls, Alarm)
(Earthquake ⟂ JohnCalls | Burglary, MaryCalls, Alarm)
(Earthquake ⟂ MaryCalls | Burglary, JohnCalls, Alarm)
(Burglary ⟂ Earthquake)
(Burglary ⟂ MaryCalls, JohnCalls | Alarm)
(Burglary ⟂ MaryCalls, JohnCalls | Earthquake,
Alarm) (Burglary ⟂ JohnCalls | MaryCalls, Alarm)
(Burglary ⟂ MaryCalls | JohnCalls, Alarm)
(Burglary ⟂ JohnCalls | Earthquake, MaryCalls,
Alarm) (Burglary ⟂ MaryCalls | Earthquake,
JohnCalls, Alarm)

Result:
Thus the program to implement a bayesian networks have been executed
successfully and the output got verified.
5. Build Regression

models Aim:

To build regression models such as locally weighted linear regression and plot
the necessary graphs.

Algorithm:

1. Read the Given data Sample to X and the curve (linear or non-linear) to Y
2. Set the value for Smoothening parameter or Free parameter say τ
3. Set the bias /Point of interest set x0 which is a subset of X
4. Determine the weight matrix using :

5. Determine the value of model term parameter β using :

6. Prediction = x0*β.

Program:

from math import ceil

import numpy as np
from scipy import
linalg

def lowess(x, y, f,
iterations): n = len(x)
r = int(ceil(f * n))
h = [np.sort(np.abs(x - x[i]))[r] for i in range(n)]
w = np.clip(np.abs((x[:, None] - x[None, :]) / h), 0.0, 1.0) w = (1 - w ** 3) ** 3
yest = np.zeros(n)
delta =
np.ones(n) for
iteration in
range(iterations): for i
in range(n):
weights = delta * w[:, i]
b = np.array([np.sum(weights * y), np.sum(weights * y * x)])
A = np.array([[np.sum(weights), np.sum(weights * x)],[np.sum(weights * x),
np.sum(weights
* x * x)]])
beta = linalg.solve(A, b)
yest[i] = beta[0] + beta[1] * x[i]

residuals = y - yest
s = np.median(np.abs(residuals))
 delta = np.clip(residuals / (6.0 * s), -1, 1)
delta = (1 - delta ** 2) **2

return yest

import math
n = 100
x = np.linspace(0, 2 * math.pi, n)
y = np.sin(x) + 0.3 *
np.random.randn(n) f =0.25
iterations=3
yest = lowess(x, y, f, iterations)

import matplotlib.pyplot as
plt plt.plot(x,y,"r.")
plt.plot(x,yest,"b-")

Output:

Result
Thus the program to implement non-parametric Locally
Weighted Regression algorithm in order to fit data points with
a graph visualization have been executed successfully.
 6. Build decision trees and random forests.

Aim:

To implement the concept of decision trees with suitable dataset from real world
problems using CART algorithm.

Algorithm:

Steps in CART algorithm:

7. It begins with the original set S as the root node.
8. On each iteration of the algorithm, it iterates through the very unused
attribute of the set S and calculates Gini index of this attribute.
9. Gini Index works with the categorical target variable “Success” or
“Failure”. It performs only Binary splits.
10. The set S is then split by the selected attribute to produce a subset of the
data.
11. The algorithm continues to recur on each subset, considering only
attributes never selected before.

Program:

import numpy as np
import matplotlib.pyplot as
plt import pandas as pd

data = pd.read_csv('Social_Network_Ads.csv')
data.head()

feature_cols = ['Age', 'EstimatedSalary']

x = data.iloc[:, [2, 3]].values
y = data.iloc[:, 4].values

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=0)

from sklearn.preprocessing import

StandardScaler sc_x = StandardScaler()
x_train = sc_x.fit_transform(x_train) x_test = sc_x.transform(x_test)

from sklearn.tree import DecisionTreeClassifier classifier =

DecisionTreeClassifier() classifier = classifier.fit(x_train, y_train) y_pred =
classifier.predict(x_test)
from sklearn import metrics
print('Accuracy Score:', metrics.accuracy_score(y_test, y_pred))

from sklearn.metrics import

confusion_matrix cm =
confusion_matrix(y_test, y_pred) print(cm)

from matplotlib.colors import ListedColormap

x_set, y_set = x_test, y_test

x1, x2 = np.meshgrid(np.arange(start=x_set[:, 0].min()-1, stop=x_set[:, 0].max()+1,

step=0.01), np.arange(start=x_set[:, 1].min()-1, stop=x_set[:, 1].max()+1, step=0.01))
plt.contourf(x1,x2, classifier.predict(np.array([x1.ravel(),
x2.ravel()]).T).reshape(x1.shape), alpha=0.75, cmap=ListedColormap(("red",
"green")))

plt.xlim(x1.min(), x1.max())
plt.ylim(x2.min(), x2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(x_set[y_set == j, 0], x_set[y_set == j, 1], c=ListedColormap(("red",
"green"))(i), label=j)

plt.title("Decision Tree(Test
set)") plt.xlabel("Age")
plt.ylabel("Estimated Salary")
plt.legend()
plt.show()

from sklearn.tree import export_graphviz

from six import StringIO
from IPython.display
import Image import
pydotplus

dot_data = StringIO()
export_graphviz(classifier, out_file=dot_data, filled=True, rounded=True,
special_characters=True, feature_names=feature_cols, class_names=['0',
'1']) graph = pydotplus.graph_from_dot_data(dot_data.getvalue())
Image(graph.write_png('decisiontree.png'))

classifier = DecisionTreeClassifier(criterion="gini", max_depth=3)

classifier = classifier.fit(x_train, y_train)
y_pred = classifier.predict(x_test)
print("Accuracy:", metrics.accuracy_score(y_test, y_pred))

dot_data = StringIO()
export_graphviz(classifier, out_file=dot_data, filled=True, rounded=True,
special_characters=True, feature_names=feature_cols, class_names=['0',
'1']) graph = pydotplus.graph_from_dot_data(dot_data.getvalue())
Image(graph.write_png('opt_decisiontree_gini.png'))
 Output of decision tree without pruning:

Optimized output of decision tree using Gini Index (CART)

Result:
Thus the program to implement the concept of decision trees with suitable dataset
from real world problems using CART algorithm have been executed successfully.
7. Build SVM models

Aim:
To create a machine learning model using Support Vector Machine algorithm.

Algorithm:

Step 1: Import the required libraries

Step 2: Load the iris dataset using the datasets.load_iris() function and store the data and
target values in variables X and y respectively.

Step 3: Create a pandas dataframe from the iris data using iris_data.data[:, [2, 3]] and
column names as iris_data.feature_names[2:].

Step 4: Split the data into training and test sets using train_test_split

Step 5: Print the number of samples in the training and test sets

Step 6: Define the markers, colors, and colormap to be used for plotting the data.

Step 7: Plot the data using a scatter plot by iterating through the unique labels and
plotting the points with the corresponding color and marker.

Step 8:Standardize the training and test data

Store the transformed data in X_train_standard and X_test_standard respectively.

Step 9: Train the SVM model using the standardized training data and the SVM() function
Store the trained model in SVM.

Step 10: Print the accuracy of the SVM model on the training and test data using
SVM.score(X_train_standard, y_train) and SVM.score(X_test_standard, y_test) respectively.

Program:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import
ListedColormap import matplotlib.pyplot as
plt
from sklearn import
datasets from sklearn.svm
import SVC
from sklearn.model_selection import
train_test_split from sklearn.preprocessing
import StandardScaler

iris_data =
datasets.load_iris() X =
ris_data.data[:, [2, 3]]
y = iris_data.target
iris_dataframe = pd.DataFrame(iris_data.data[:, [2,
3]], columns=iris_data.feature_names[2:])
print(iris_dataframe.head())
print('\n' + 'Unique Labels contained in this data are ' + str(np.unique(y)))
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
print('The training set contains {} samples and the test set contains {}
samples'.format(X_train.shape[0], X_test.shape[0]))
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
print('The training set contains {} samples and the test set contains {}
samples'.format(X_train.shape[0], X_test.shape[0]))
markers = ('x', 's', 'o')
colors = ('red', 'blue', 'green')
cmap = ListedColormap(colors[:len(np.unique(y_test))])

for idx, cl in enumerate(np.unique(y)):

plt.scatter(x=X[y == cl, 0], y=X[y == cl, 1],
c=cmap(idx), marker=markers[idx], label=cl)
standard_scaler = StandardScaler()
standard_scaler.fit(X_train)
X_train_standard = standard_scaler.transform(X_train)
X_test_standard = standard_scaler.transform(X_test)
print('The first five rows after standardisation look like this:\n')
print(pd.DataFrame(X_train_standard,
columns=iris_dataframe.columns).head()) SVM = SVC(kernel='rbf',
random_state=0, gamma=.10, C=1.0) SVM.fit(X_train_standard, y_train)
print('Accuracy of our SVM model on the training data is',(SVM.score(X_train_standard,
y_train)))
print('Accuracy of our SVM model on the test data is',(SVM.score(X_test_standard, y_test)))

Output:
petal length (cm) petal width (cm)
0 1.4 0.2
1 1.4 0.2
2 1.3 0.2
3 1.5 0.2
4 1.4 0.2

Unique Labels contained in this data are [0 1 2]

The training set contains 105 samples and the test set contains 45
samples The training set contains 105 samples and the test set contains
45 samples The first five rows after standardisation look like this:

petal length (cm) petal width (cm)

0 -0.182950 -0.293181
1 0.930661 0.737246
2 1.042022 1.638870
3 0.652258 0.350836
4 1.097702 0.737246
Accuracy of our SVM model on the training data is 0.9523809523809523
Accuracy of our SVM model on the test data is 0.9777777777777777
Result:

Thus the machine learning model was created using Support Vector Machine
algorithm.
8. Implement ensembling techniques

Aim:

To implement the ensembling technique of Blending with the given

Alcohol QCM Dataset.

Algorithm:

1. Split the training dataset into train, test and validation dataset.
2. Fit all the base models using train dataset.
3. Make predictions on validation and test dataset.
4. These predictions are used as features to build a second level model
5. This model is used to make predictions on test and meta-features.

Program:

import pandas as pd
from sklearn.metrics import
mean_squared_error from sklearn.ensemble
import RandomForestRegressor import
xgboost as xgb from sklearn.linear_model
import LinearRegression f
rom sklearn.model_selection import
train_test_split df = pd.read_csv("train_data.csv")
target = df["target"] train = df.drop("target")
X_train, X_test, y_train, y_test = train_test_split(train, target, test_size=0.20)
train_ratio = 0.70
validation_ratio = 0.20
test_ratio = 0.10
x_train, x_test, y_train, y_test = train_test_split( train, target, test_size=1 - train_ratio)
x_val, x_test, y_val, y_test = train_test_split(
x_test, y_test, test_size=test_ratio/(test_ratio +
validation_ratio)) model_1 = LinearRegression()
model_2 = xgb.XGBRegressor()
model_3
=RandomForestRegressor()
model_1.fit(x_train, y_train)
val_pred_1 = model_1.predict(x_val)
test_pred_1 =
model_1.predict(x_test)
val_pred_1 = pd.DataFrame(val_pred_1)
test_pred_1 = pd.DataFrame(test_pred_1)
model_2.fit(x_train, y_train)
val_pred_2 = model_2.predict(x_val)
test_pred_2 = model_2.predict(x_test)
val_pred_2 = pd.DataFrame(val_pred_2)
test_pred_2 = pd.DataFrame(test_pred_2)
model_3.fit(x_train, y_train)
val_pred_3 = model_1.predict(x_val)
test_pred_3 = model_1.predict(x_test)
val_pred_3 = pd.DataFrame(val_pred_3) test_pred_3
= pd.DataFrame(test_pred_3)
df_val = pd.concat([x_val, val_pred_1, val_pred_2, val_pred_3], axis=1)
df_test = pd.concat([x_test, test_pred_1, test_pred_2, test_pred_3],
axis=1) final_model = LinearRegression()
final_model.fit(df_val, y_val)
final_pred = final_model.predict(df_test)
print(mean_squared_error(y_test, pred_final))

Output:

4790

Result:
Thus the program to implement ensembling technique of Blending with
the given Alcohol QCM Dataset have been executed successfully and the
output got verfied.
9. Implement clustering algorithms

Aim:

To implment k-Nearest Neighbour algorithm to classify the Iris Dataset.

Algorithm:

Step 1: Import necessary libraries

Step 2: Load iris dataset using datasets.load_iris() function and create a pandas
dataframe named 'x' to store the data.
Step 3: Create a new pandas dataframe named 'y' to store the target variable.
Step 4: Create a scatter plot with two subplots
Step 5: Create a KMeans model named 'iris_k_mean_model' with 3 clusters using
KMeans(n_clusters=3)
Step 6: Fit the KMeans model on the iris data using iris_k_mean_model.fit(x)
Step 7: Print the cluster centers of the fitted KMeans model using
iris_k_mean_model.cluster_centers_

Program:

import pandas as pd
import numpy as np
from sklearn import datasets
from sklearn.cluster import
KMeans import matplotlib.pyplot
as plt
import matplotlib.patches as
mpatches import sklearn.metrics as
sm
iris = datasets.load_iris()
x = pd.DataFrame(iris.data, columns=['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal
Width'])
y = pd.DataFrame(iris.target, columns=['Target'])
plt.figure(figsize=(12,3))
colors = np.array(['red', 'green', 'blue'])
iris_targets_legend = np.array(iris.target_names)
red_patch = mpatches.Patch(color='red',
label='Setosa')
green_patch = mpatches.Patch(color='green', label='Versicolor')
blue_patch = mpatches.Patch(color='blue', label='Virginica')
plt.subplot(1, 2, 1)
plt.scatter(x['Sepal Length'], x['Sepal Width'], c=colors[y['Target']])
plt.title('Sepal Length vs Sepal Width')
plt.legend(handles=[red_patch, green_patch, blue_patch])
plt.subplot(1,2,2)
plt.scatter(x['Petal Length'], x['Petal Width'], c=
colors[y['Target']]) plt.title('Petal Length vs Petal Width')
plt.legend(handles=[red_patch, green_patch, blue_patch])
iris_k_mean_model = KMeans(n_clusters=3)
iris_k_mean_model.fit(x)
print (iris_k_mean_model.cluster_centers_)
Result:
Thus the program to implement k-Nearest Neighbour Algorithm
for clustering Iris dataset have been executed successfully and
output got verified.
10. Implement EM for Bayesian networks

Aim:

To implement the EM algorithm for clustering networks using the given dataset.

Algorithm:

Step 1: Initialize θ randomly Repeat until

convergence: Step 2: E-step:
Compute q(h) = P(H = h | E = e; θ) for each h (probabilistic
inference) Create fully-observed weighted examples: (h, e) with
weight q(h) Step 3: M-step:
Maximum likelihood (count and normalize) on weighted examples to get θ

Program:

from sklearn.cluster import

KMeans from sklearn import
preprocessing
from sklearn.mixture import
GaussianMixture from sklearn.datasets
import load_iris
import sklearn.metrics as sm
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
dataset=load_iris()
X=pd.DataFrame(dataset.data)
X.columns=['Sepal_Length','Sepal_Width','Petal_Length','Petal_Width']
y=pd.DataFrame(dataset.target)
y.columns=['Targets']
plt.figure(figsize=(14,7))
colormap=np.array(['red','lime','black'])

# REAL PLOT
plt.subplot(1,3,1)
plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[y.Targets],s=4
0) plt.title('Real')

# K-PLOT
plt.subplot(1,3,2)
model=KMeans(n_clusters=3)
model.fit(X)
predY=np.choose(model.labels_,[0,1,2]).astype(np.int64)
plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[predY],s=40)
plt.title('KMeans')

# GMM PLOT
scaler=preprocessing.StandardScaler()
scaler.fit(X)
xsa=scaler.transform(X)
xs=pd.DataFrame(xsa,columns=X.columns)
gmm=GaussianMixture(n_components=3)
gmm.fit(xs)
y_cluster_gmm=gmm.predict(xs)
plt.subplot(1,3,3)
plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[y_cluster_gmm],s=40)
plt.title('GMM Classification')

Output:
Text(0.5, 1.0, 'GMM Classification')

Result:
Thus the program for Expectation Maximization Algorithm was executed and verified.
11. Build simple NN models

Aim :
To implement the neural network model for the given numpy array.

Algorithm:
Step 1: Use numpy arrays to store inputs (x) and outputs (y)
Step 2: Define the network model and its arguments.
Step 3: Set the number of neurons/nodes for each
layer Step 4: Compile the model and calculate its
accuracy Step 5: Print a summary of the Keras model

Program:
from keras.models import Sequential
from keras.layers import Dense,
Activation import numpy as np
x = np.array([[0,0], [0,1], [1,0], [1,1]])
y = np.array([[0], [1], [1], [0]])
model = Sequential()
model.add(Dense(2, input_shape=(2,)))
model.add(Activation('sigmoid'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='mean_squared_error', optimizer='sgd', metrics=['accuracy'])
model.summary()

Output:

Result: Thus the program to implement the neural network model for the given dataset.
12. Build deep learning NN models

Aim:

To implement and build a Convolutional neural network model which predicts

the age and gender of a person using the given pre-trained models.

Algorithm:

Step-1: Import the required

packages Step-2: Load the MNIST
dataset
Step-3: Normalize the training and test
data Step-4: Visualize the normalized first
image in the training dataset
Step-5: Define the model
architecture Step-6: Compile the
model
Step-7: Train the model
Step-8: Evaluate the model on the test data
Model. Step-9: Save the model
Step 10: Load the saved model
Step 11: Make predictions on the test data using
the loaded model
Step 12: Visualize the first test image and
its predicted label

Program:

import tensorflow.keras as keras

import tensorflow as tf
print(tf. version )
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) =
mnist.load_data() import matplotlib.pyplot as plt

plt.imshow(x_train[0],cmap=plt.cm.binary)
plt.show()
print(y_train[0])
x_train = tf.keras.utils.normalize(x_train, axis=1)
x_test = tf.keras.utils.normalize(x_test, axis=1)
print(x_train[0])

plt.imshow(x_train[0],cmap=plt.cm.binary)
plt.show()
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(10, activation=tf.nn.softmax))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3)
val_loss, val_acc = model.evaluate(x_test,
y_test) print(val_loss)
print(val_acc)
model.save('epic_num_reader.model')
new_model = tf.keras.models.load_model('epic_num_reader.model')
predictions = new_model.predict(x_test)
import numpy as np

print(np.argmax(predictions[0]))
plt.imshow(x_test[0],cmap=plt.cm.binary)
plt.show()

Result:
Thus the program to implement and build a Convolutional neural network model
which have been executed successfully and the output got verified.