Machine Learning Techniques

UNIT-01

Ms. Amrita Rai

Assistant Professor
Computer Science & Engineering(MLT)
Definition of Learning
Learning involves changes in behavior due to experiences,
focusing on adapting rather than relying on instinct or temporary
states.
1

Types of Learning in Machine Learning

Machine learning can be broadly classified into three
main types:

1. Supervised Learning Definition:

• In supervised learning, the model is trained on a
labeled dataset, which means that each input data
point has a corresponding correct output (label).
• The model learns by comparing its predicted output
with the actual output and adjusts to reduce errors.

Example:
• Image Classification: Given a dataset of images
labeled as "cats" or "dogs," the model learns to classify
new images as either a cat or a dog.
1

Algorithm Examples:
• Linear Regression
• Support Vector Machines (SVM)
• Decision Trees
• Neural Networks

2. Unsupervised Learning Definition:

• In unsupervised learning, the model is trained on an
unlabeled dataset.
• The task is to identify patterns, clusters, or
associations within the data without any explicit
guidance on what the output should be.
Example:
• Customer Segmentation: An e-commerce company
might want to segment customers into different groups
based on their purchasing behavior. The algorithm
identifies groups with similar behavior, even though no
labels are provided.
1

Algorithm Examples:
• K-means Clustering
• Principal Component Analysis (PCA)
• Hierarchical Clustering

3. Semi-Supervised Learning
• Semi-supervised learning is a mix between supervised
and unsupervised learning.
• It uses a small amount of labeled data and a large
amount of unlabeled data. The model improves by
learning from both.

Example: A small set of labeled emails as spam or not,

combined with a large set of unlabeled emails, to
improve the spam detection model.
1

4. Reinforcement Learning:
• In reinforcement learning, an agent interacts with an
environment and learns to take actions that maximize
cumulative rewards.
• Instead of being trained on labeled data, the agent
learns by trial and error, receiving rewards or penalties
based on the actions taken.

Example:
• Game Playing: In a game like chess or Go, the
algorithm learns to play by receiving positive rewards for
winning and negative rewards for losing, optimizing its
strategy over time.

Algorithm Examples:
• Q-Learning
• Deep Q Networks (DQN)
I

Well-defined learning problems

A well-defined learning problem allows a computer program to

improve at a specific task through experience. This is characterized
by three key elements:

Task (T): The specific activity or challenge the program is expected

to perform.

Performance Measure (P): The criteria used to gauge the

program's effectiveness at the task.

Experience (E): The data or interactions from which the program

learns.
1

Machine learning
• Machine learning is a subset of artificial intelligence (Al) that
enables computers to learn from and make decisions based on
data, without being explicitly programmed.

Definition: Machine learning involves developing algorithms that

allow computers to process and learn from data automatically.

Purpose: The aim is to enable computers to learn from their

experiences and improve their performance over time without
human intervention.

Functionality: Machine learning algorithms analyze vast amounts

of data, enabling them to perform tasks more efficiently and
accurately. This could be anything from predicting consumer
behavior to detecting fraudulent transactions.

Integration: Combining machine learning with Al and cognitive

technologies enhances its ability to process and interpret large
volumes of complex data.
Advantages of Machine Learning
• Identifies trends and patterns
• Automation
• Continuous Improvement
• Handling Complex data

Disadvantages of Machine Learning

• Data Acquisition
• Interpretation of result
• High error- Susceptibility
Overview of the history of Machine Learning
Early Development
• 1943: Neurophysiologist Warren McCulloch and
mathematician Walter Pitts introduced the concept of
a neural network by modeling neurons with electrical
circuits.
2

• 1952: Arthur Samuel developed the first computer

program capable of learning from its activities.
• 1958: Frank Rosenblatt created the Perceptron, the
first artificial neural network, which was designed for
pattern and shape recognition.
• 1959: Bernard Widrow and Marcian Hoff developed
two neural network models: ADELINE, which could
detect binary patterns, and MADELINE, which was
used to reduce echo on phone lines.
• Advancements in the 1980s and 1990s: 1982:
John Hopfield proposed a network with bidirectional
lines that mimicked actual neuronal structures.
8

• 1986: The backpropagation algorithm was

popularized, allowing the use of multiple layers in
neural networks, enhancing their learning
capabilities.
• 1997: IBM's Deep Blue, a chess-playing computer,
famously beat the reigning world chess champion.
• 1998: AT&T Bell Laboratories achieved significant
progress in digit recognition, notably enhancing the
ability to recognize handwritten postcodes for the US
Postal Service.
• 21st Century Innovations: The 21st century
has seen a significant surge in machine learning,
driven by both industry and academia, to boost
computational capabilities and innovation.
Machine Learning Approaches
Artificial Neural Network (ANN): An Artificial
Neural Network (ANN) is a computational model
inspired by the way biological neural networks in the
human brain work.
• It is designed to recognize patterns, learn from data,
and make decisions or predictions by mimicking the
structure and functioning of neurons in a brain.
• ANNs are the core technology behind deep learning
and are widely used in applications such as image
recognition, speech processing, and more.

Key Components of an ANN:

1. Neurons (Nodes): Basic units of the network. Each
neuron takes input, processes it, and generates an
output.
1

2. Layers:
o Input Layer: Receives the initial data and passes it to
the hidden layer.
o Hidden Layers: Perform computations on the input
data. There can be multiple hidden layers in deep neural
networks.
o Output Layer: Produces the final prediction or output
of the network.

[Link] and Biases: Weights control the strength of

the connection between neurons. Bias helps adjust the
output along with weights.

4. Activation Function: Introduces non-linearity to the

model, allowing the network to learn complex patterns.
Common activation functions include sigmoid, and ReLU.
 1

1
1

Clustering
• Clustering is a type of unsupervised machine learning
technique used to group a set of objects or data points
into clusters based on their similarities or patterns.
• The goal is to ensure that objects within the same
cluster are more similar to each other than to those in
other clusters.

Example:
Customer Segmentation:
In a retail business, a company may want to understand its
customer base better to tailor marketing strategies. By
analyzing customer data (such as age, income, purchasing
behavior, and shopping frequency), clustering can be
employed to group customers into distinct segments.
1

It consists of the following steps

1. Data Collection: Gather customer data, which may
include variables like age, income, purchase history,
and more.

[Link] Algorithm: Use a clustering algorithm like

K-means, hierarchical clustering, or DBSCAN to identify
clusters.

[Link]: After applying the algorithm, the company may

find several clusters, such as:

 Cluster 1: Young, high-income customers who

frequently shop online.
 Cluster 2: Middle-aged customers who prefer in-store
shopping.
 Cluster 3: Older customers with low purchase
frequency.
1

Data points before clustering

1

Reinforcement Learning
• In reinforcement learning, an agent interacts with an
environment and learns to take actions that maximize
cumulative rewards.
• Instead of being trained on labeled data, the agent
learns by trial and error, receiving rewards or penalties
based on the actions taken.

Example:
• Game Playing: In a game like chess or Go, the
algorithm learns to play by receiving positive rewards for
winning and negative rewards for losing, optimizing its
strategy over time.

Algorithm Examples:
• Q-Learning
• Deep Q Networks (DQN)
1

Decision Tree Learning

• Decision Tree Learning is a supervised machine learning
technique used for classification.
• It involves creating a model that predicts the value of a target
variable based on several input features.
• The model is represented as a tree structure, where each internal
node represents a decision based on an attribute, each branch
represents the outcome of that decision, and each leaf node
represents a final prediction or class label.
 1

Support Vector Machine (SVM)

• Support Vector Machine (SVM) is a supervised machine learning
algorithm primarily used for classification tasks, but it can also be
adapted for regression.

• SVM aims to find the optimal hyperplane that separates data points of
different classes in a high-dimensional space.

• The optimal hyperplane is the one that maximizes the margin, which is
the distance between the hyperplane and the nearest data points from
each class, known as support vectors.
1

Genetic Algorithm
Example of a Genetic Algorithm:
Scenario: Imagine you want to find the best route for a delivery truck that needs to
visit several cities.
1. Step 1 - Population: Start with random routes, like:
• Route 1: City A → City B → City C → City D
• Route 2: City C → City A → City D → City B
2. Step 2 - Fitness Function: Calculate the total distance for each route. Shorter
distances score higher. For example: Route 1: 100 miles
Route 2: 80 miles (better route)
1

3. Step 3 - Selection: Select the best routes based on distance. In this

case, Route 2 is chosen more often.

4. Step 4 - Crossover: Mix routes to create new ones. If you combine

parts of Route 1 and Route 2, you might get:
o New Route: City C → City B → City A → City D

5. Step 5 - Mutation: Randomly change a route by swapping two cities.

For example:
• From: City C → City B → City A → City D
• To: City C → City A → City B → City D

6. Step 6 - Repeat: Keep repeating the process of evaluating, selecting,

crossing over, and mutating until you find the shortest route.
1

Bayesian Belief Networks (BBNs)

• A Bayesian Belief Network (BBN), also called a Bayesian
Network or Bayes Network, is a probabilistic graphical
model that represents a set of variables and their
conditional dependencies using a directed acyclic graph
(DAG).
• It provides a way to model uncertainty by describing the
joint probability distribution among the variables, and it
allows reasoning about the likelihood of various events
based on the known information.

Key Components:
1. Nodes: Represent random variables (these could be
observed variables, latent variables, or hypotheses).
2. Edges (Arrows): Represent conditional dependencies
between the variables. An edge from node A to node B
means that B is conditionally dependent on A.
.
1

[Link] Probability Tables (CPTs): Each node

has a conditional probability distribution that quantifies
the effect of its parent nodes (if any) on that variable.

Example of a Bayesian Belief Network:

Consider a simple Bayesian network that models the
relationship between three variables:
[Link] (W): Can be "Sunny", "Rainy", or "Cloudy".
[Link] (S): Whether a sprinkler was turned on
(Yes/No).
[Link] Grass (G): Whether the grass is wet (Yes/No).

Structure:
• Weather influences both the Sprinkler and whether the
Grass is Wet.
• Sprinkler also influences whether the Grass is Wet.
1

Network structure would look like this:

1

Issues in Machine Learning

1. Data Quality and Quantity
• Insufficient Data: Many machine learning models
require large amounts of data to learn effectively. A lack
of data can lead to overfitting, where the model performs
well on training data but poorly on unseen data.

• Noisy Data: Data can be noisy or contain errors,

which can negatively affect the model's performance.

• Imbalanced Data: When the dataset has an unequal

distribution of classes (e.g., many more examples of one
class than another), it can lead to biased models that
perform poorly on the minority class.
1

[Link] Overfitting and Under fitting

• Overfitting: A model that learns the training data too
well may cap2ture noise instead of the underlying
patterns, resulting in poor generalization to new data.
• Under fitting: A model that is too simple may fail to
capture important relationships in the data, leading to
poor performance on both training and test datasets.

3. Feature Engineering
• Feature Selection: Identifying the most relevant
features to use for model training can be challenging and
time-consuming.

•Dimensionality Reduction: High-dimensional data

can lead to the "curse of dimensionality," where the
model struggles to learn effectively due to the sparsity of
data in high dimensions.
1

Data Science Vs Machine

Learning
1

Traditional Programming Vs Machine

Learning

Traditional Programming
• Definition: In traditional programming, developers
write explicit instructions (code) that tell the computer
how to perform a specific task.
• The program follows these rules to produce the desired
output.

Example: A Simple Calculator

• Task: Perform basic arithmetic operations (addition,
subtraction, multiplication, division).

• Implementation: A programmer writes a function that

takes two numbers and an operation as input and returns
the result. The logic is predefined.
1

Machine Learning
• Definition: In machine learning, the program learns
from data instead of following explicit instructions.
• The developer provides a dataset, and the algorithm
identifies patterns within the data to make predictions
or decisions.

Example: Email Spam Detection

• Task: Classify emails as "spam" or "not spam."
• Implementation: A machine learning model is trained
on a dataset of labeled emails (spam and not spam).
The model learns features that distinguish spam from
non-spam emails.
Thank
you