Lecture 17
Course Outcome: Apply the concept of machine learning algorithms including neural networks and
supervised/unsupervised learning techniques for engineering applications.
Recap
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 2 / 34
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 3 / 34
What is Supervised Learning?
Supervised learning is a type of machine learning where the algorithm
learns from a labeled dataset.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 4 / 34
What is Supervised Learning?
Supervised learning is a type of machine learning where the algorithm
learns from a labeled dataset.
Definition: Labeled Data
Labeled data refers to datasets where each input is paired with a correct
output label. This label represents the ”ground truth” or the right answer
for that input.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 4 / 34
Example
Labeled Data of Fruits
Imagine a dataset of fruit images.
An image of an apple (the input) is tagged with the label “apple” (the output).
An image of an orange (the input) is tagged with the label “orange” (the output).
The model uses these labeled examples to learn the visual differences.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 5 / 34
Formal Definition
Definition 17.1: Supervised Learning
Supervised learning is a machine learning paradigm in which a model is trained on a labeled
dataset to learn a mapping function that can predict the output for new, unseen inputs.
Key Characteristics
✓ Requires labeled data for the training phase.
✓ The primary goal is to learn a function that maps inputs to desired outputs.
✓ Can be used for two main types of tasks:
Classification: Predicting distinct categories (e.g., ”spam” or ”not spam”).
Regression: Predicting continuous numerical values (e.g., house prices).
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 6 / 34
The Building Blocks
In any supervised learning problem, we have three core components:
Input (Features): The data we give the Example: Spam Email Filter
model. This could be images, text, sensor Input: The text content, sender, and
readings, or numerical values. other features of thousands of emails.
Output (Labels): The target we want to Output: A label for each email: “Spam”
predict. These are the correct answers in or “Not Spam”.
the training data.
Goal: Train a model that accurately
Goal: To minimize the error between the classifies new, incoming emails it has
model’s predictions and the actual labels, never seen before.
especially on new data.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 7 / 34
Considerations
A Major Challenge: The Cost of Labeling
Supervised learning is highly effective, but its performance depends entirely on the quality
and quantity of labeled data.
Creating large, accurately labeled datasets can be:
Time-consuming: Manually labeling thousands or millions of data points takes time.
Expensive: Often requires hiring domain experts (e.g., doctors to label medical
images, lawyers to classify legal documents).
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 8 / 34
The Two Types of Supervised Learning
Supervised learning tasks are primarily divided into two major categories based on the kind of
output they predict.
Classification Regression
Predicts a Category or Class Label. Predicts a Continuous Numerical Value.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 9 / 34
Types of Supervised Learning: Classification
Definition 17.2: Definition: Classification
Classification is a supervised learning task where the goal is to predict a category or a
discrete class label. The output is not a number, but a group assignment.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 10 / 34
Types of Supervised Learning: Classification
Definition 17.3: Definition: Classification
Classification is a supervised learning task where the goal is to predict a category or a
discrete class label. The output is not a number, but a group assignment.
Common Questions Classification Can Answer
Is this transaction “fraudulent” or “not fraudulent”?
Is this a spam email or a non-spam email?
What grade will I get for GNEST130?
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 10 / 34
Case Study
The Titanic Dataset
The goal is to predict if a passenger survived (1) or not (0) based on their data. This is a
classic binary classification problem.
Pclass Sex Age SibSp Parch Fare Cabin Embarked Survived
3 male 22.0 1 0 7.2500 – S 0
1 female 38.0 1 0 71.2833 C85 C 1
3 female 26.0 0 0 7.9250 – S 1
1 female 35.0 1 0 53.1000 C123 S 1
3 male 35.0 0 0 8.0500 – S 0
The highlighted Survived column is the label we want to predict.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 11 / 34
Types of Supervised Learning: Regression
Definition 17.4: Definition: Regression
Regression is a supervised learning task where the goal is to predict a continuous numer-
ical value. The output can be any number within a given range.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 12 / 34
Types of Supervised Learning: Regression
Definition 17.5: Definition: Regression
Regression is a supervised learning task where the goal is to predict a continuous numer-
ical value. The output can be any number within a given range.
Common Questions Regression Can Answer
What is the predicted price of this house?
How many sales will we make next quarter?
What will the temperature be tomorrow?
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 12 / 34
Summary: Classification vs. Regression
A Side-by-Side Comparison
Feature Classification Regression
Primary Goal To predict a discrete class label To predict a continuous numer-
or category. ical quantity.
Output Type Categorical. The output is a Numerical. The output is a
class from a finite set of possi- real value that can be mea-
bilities. sured.
Question Answered ”What kind is it?” or ”To ”How much?” or ”How
which group does it belong?” many?”
Examples
Is an email Spam/Not What is the price of this
Spam? house?
What animal is in this im- What will the temperature
age? be?
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 13 / 34
Test Your Intuition: Regression or Classification?
1. Predicting if an online review is positive, neutral, or negative.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 14 / 34
Test Your Intuition: Regression or Classification?
1. Predicting if an online review is positive, neutral, or negative.
2. You are building a system to generate a credit score between 300 and 850 for a loan
applicant.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 14 / 34
Test Your Intuition: Regression or Classification?
1. Predicting if an online review is positive, neutral, or negative.
2. You are building a system to generate a credit score between 300 and 850 for a loan
applicant.
3. Predicting the probability that a patient has diabetes.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 14 / 34
Test Your Intuition: Regression or Classification?
1. Predicting if an online review is positive, neutral, or negative.
2. You are building a system to generate a credit score between 300 and 850 for a loan
applicant.
3. Predicting the probability that a patient has diabetes.
4. You are building a system to estimate a person’s age in years from a profile picture.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 14 / 34
Test Your Intuition: Regression or Classification?
1. Predicting if an online review is positive, neutral, or negative.
2. You are building a system to generate a credit score between 300 and 850 for a loan
applicant.
3. Predicting the probability that a patient has diabetes.
4. You are building a system to estimate a person’s age in years from a profile picture.
5. You are building a system to predict the clinical stage of a disease (I, II, III, or IV)
based on medical scans.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 14 / 34
Unsupervised Learning
Unsupervised learning is a type of machine learning where the algorithm explores patterns in
unlabeled data.
Unlike supervised learning, there is no predefined target or output label.
The goal is to uncover hidden structures, groupings, or lower-dimensional representations
within the data.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 15 / 34
Definition of Unsupervised Learning
Definition 17.6: Unsupervised Learning
Unsupervised learning is a machine learning approach where models are trained on data
that has no labeled outputs. The system attempts to learn the underlying structure or
distribution in the data to discover useful patterns or groupings.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 16 / 34
Analogy: How It Works
Unsupervised Learning
As part of your internship, you visit a university in
Germany. Over the weekend, you’re out shopping,
but you don’t know German, so you can’t read the
signs on the stores.
However, by observing the items displayed—like
vegetables, clothes, electronics—you start group-
ing shops based on what they appear to sell.
You didn’t have any labels, but you still formed
meaningful clusters.
That’s what unsupervised learning does—it helps
us find structure and patterns in data without any
predefined categories.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 17 / 34
Examples of Unsupervised Learning Tasks
Grouping customers with similar behavior (clustering).
Visualizing high-dimensional data in 2D or 3D space (dimensionality reduction).
Detecting anomalies or outliers in transaction data.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 18 / 34
Clustering
Clustering is the process of dividing a dataset into
groups, or clusters, such that data points in the
same group are more similar to each other than to
those in other groups.
Definition 17.7: Clustering
Clustering is an unsupervised learning technique
used to group similar data points together based
on some notion of distance or similarity.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 19 / 34
Dimensionality Reduction
Dimensionality reduction is the process of reducing the number of features (dimensions) in a
dataset while preserving as much information as possible. It is often used for:
Data visualization
Noise removal
Improving model performance
Definition 17.8: Dimensionality Reduction
Dimensionality reduction techniques transform high-dimensional data into a lower-
dimensional space while preserving meaningful properties like variance or class separability.
Key techniques include:
Principal Component Analysis (PCA)
Singular Value Decomposition (SVD)
Dimensionality reduction helps to address the curse of dimensionality.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 20 / 34
The Curse of Dimensionality
As the number of dimensions (features) increases, data becomes sparse.
Distance measures (like Euclidean distance) lose meaning in high dimensions.
Models require exponentially more data to generalize well.
Finding neighbors in high dimensions is like looking for a friend in a massive stadium —
everyone seems far apart.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 21 / 34
Distance Concentration Problem
In high dimensions, the difference between the nearest and farthest neighbor distances
shrinks.
This makes clustering and nearest-neighbor methods unreliable.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 22 / 34
Why Dimensionality Reduction Helps
Removes redundant/noisy features.
Captures essential structure in fewer dimensions.
Makes clustering, visualization, and classification feasible.
Dimensionality reduction is the antidote to the curse of dimensionality.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 23 / 34
Clustering vs. Dimensionality Reduction
Clustering tells us “who belongs together,” while dimensionality reduction tells us “what
really matters” in the data.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 24 / 34
Clustering vs. Dimensionality Reduction (1)
Aspect Clustering Dimensionality Reduc-
tion
Definition Grouping data points Reducing the number of
into clusters based on input variables while pre-
similarity serving important infor-
mation
Goal Discover hidden group- Simplify data representa-
ings or patterns in data tion, reduce noise and re-
dundancy
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 25 / 34
Clustering vs. Dimensionality Reduction (2)
Aspect Clustering Dimensionality Reduc-
tion
Output Cluster labels or assign- Transformed features in a
ments for each data lower-dimensional space
point
Target Variable Not used (unsupervised) Not used (unsupervised
or semi-supervised in
some methods)
Common Algo- K-Means, DBSCAN, PCA, t-SNE, UMAP,
rithms Hierarchical Clustering, LDA, Autoencoders
Gaussian Mixture Models
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 26 / 34
Clustering vs. Dimensionality Reduction (3)
Aspect Clustering Dimensionality Reduc-
tion
Evaluation Met- Silhouette score, Explained variance ratio,
rics Davies–Bouldin in- reconstruction error, vi-
dex, visual inspection sualization quality
Supervision Unsupervised Mostly unsupervised
(LDA is supervised)
Main Use Cases Market segmentation, Data visualization,
anomaly detection, so- speeding up model train-
cial network analysis ing, noise filtering
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 27 / 34
Clustering vs. Dimensionality Reduction (4)
Aspect Clustering Dimensionality Reduc-
tion
Visual Represen- Scatter plots with cluster PCA plots, 2D/3D
tation coloring, dendrograms embeddings of high-
dimensional data
Feature Use Uses original features to Constructs new features
measure similarity or projections from orig-
inal data
Interpretability Clusters may be intuitive Principal components or
but often need domain embeddings may lack in-
interpretation tuitive meaning
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 28 / 34
Clustering vs. Dimensionality Reduction (5)
Aspect Clustering Dimensionality Reduc-
tion
Data Transfor- No transformation; out- Transforms data into a
mation puts cluster indices lower-dimensional repre-
sentation
Dimensionality High dimensions may Aims to solve the curse of
Impact hinder performance due dimensionality
to sparsity
Reconstruction Not designed to recon- Some methods (e.g.,
Ability struct original data PCA, autoencoders) can
reconstruct input data
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 29 / 34
Clustering vs. Dimensionality Reduction (6)
Aspect Clustering Dimensionality Reduc-
tion
Probabilistic Some algorithms (e.g., Some methods (e.g.,
View GMM) provide soft clus- PCA) provide variance-
ter probabilities based interpretations
Example Task Grouping customers by Reducing 1000-
purchasing behavior dimensional image
data to 2D for visualiza-
tion
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 30 / 34
Supervised vs. Unsupervised Learning (1/3)
Aspect Supervised Learning Unsupervised Learning
Definition Learns a mapping from Discovers hidden pat-
input to output using la- terns or structures from
beled data unlabeled data
Data Require- Requires labeled data Requires only input data
ment (input-output pairs) (no labels)
Objective Predict output values or Group, cluster, or com-
class labels for new in- press data by finding pat-
puts terns
Target Variable Present Not present
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 31 / 34
Supervised vs. Unsupervised Learning (2/3)
Aspect Supervised Learning Unsupervised Learning
Examples of Al- Linear/Logistic Regres- K-Means, DBSCAN,
gorithms sion, SVM, Decision Hierarchical Clustering,
Trees, k-NN, Neural PCA
Networks
Output Type Discrete labels or contin- Clusters, embeddings, or
uous values reduced features
Evaluation Met- Accuracy, Precision, Re- Silhouette score,
rics call, F1, MSE, R 2 Davies–Bouldin in-
dex, visualization
Common Tasks Spam detection, fraud Customer segmentation,
detection, stock price anomaly detection, topic
prediction modeling
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 32 / 34
Supervised vs. Unsupervised Learning (3/3)
Aspect Supervised Learning Unsupervised Learning
Labeling Cost High (needs annotated Low (no manual labels)
data)
Human Inter- Easier to validate with Harder, often exploratory
pretability ground truth
Visualization Confusion matrix, ROC Cluster plots, dimension-
Tools curves, residual plots ality reduction plots
Example Task Predict if a customer will Group customers based
default on a loan on purchasing behavior
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 33 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
2. Grouping news articles into topics when no categories are given.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
2. Grouping news articles into topics when no categories are given.
3. Predicting the price of a house given features like area and location.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
2. Grouping news articles into topics when no categories are given.
3. Predicting the price of a house given features like area and location.
4. Clustering students into study groups based on their course activity, without labels.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
2. Grouping news articles into topics when no categories are given.
3. Predicting the price of a house given features like area and location.
4. Clustering students into study groups based on their course activity, without labels.
5. Classifying handwritten digits (0–9) from labeled images.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
Test Your Intuition: Supervised or Unsupervised?
1. Predicting if an email is Spam or Not Spam using labeled examples.
2. Grouping news articles into topics when no categories are given.
3. Predicting the price of a house given features like area and location.
4. Clustering students into study groups based on their course activity, without labels.
5. Classifying handwritten digits (0–9) from labeled images.
6. Reducing a dataset of 100 features into 2D for visualization.
Dr. Jerrin Thomas Panachakel (CET) Introduction to Artificial Intelligence and Data Science 34 / 34
