BCS602 (Machine Learning)

MACHINE LEARNING
Module-1
Chapter – 01 – Introduction

Need for Machine Learning in Business:

Data Explosion and Integration Challenges:

o Businesses generate massive amounts of data daily.

o Earlier, the potential of this data was not utilized due to scattered data sources and lack of
integration tools.

Reasons for Popularity of Machine Learning:

o High Volume of Data:

 Companies like Facebook, Twitter, and YouTube generate enormous data, which doubles
approximately every year.

o Reduced Storage Costs:

 Declining hardware and storage costs make it easier to capture, store, process, and
distribute digital data.

o Availability of Complex Algorithms:

 The development of advanced algorithms, particularly deep learning, has contributed to

the growing popularity of machine learning.

Knowledge Pyramid:

o Data: Raw facts, numbers, or text that can be processed by computers.

o Information: Processed data revealing patterns or relationships, e.g., identifying the best-selling
product from sales data.

o Knowledge: Condensed and actionable information, such as historical trends or future

predictions from sales data.

o Intelligence: Applied knowledge for making decisions and taking actions.

o Wisdom: The ultimate objective, representing the maturity of human decision- making, which
machine learning helps organizations to approach.
 BCS602 (Machine Learning)

Role of Machine Learning:

o Machine learning helps businesses process large amounts of data to make informed decisions.

o It aids in designing new products, improving business processes, and developing effective
decision support systems.

Machine Learning Explained

Definition of Machine Learning:

 Arthur Samuel's Definition: "Machine learning gives computers the ability to learn without being
explicitly programmed."

 Focus: Systems learn autonomously by analyzing data without manual programming.

Conventional Programming vs. Machine Learning:

Conventional Programming:

o Requires explicit instructions and logic.

o Involves detailed program design like flowcharts and algorithms.

Limitations:

o Ineffective for complex problems (e.g., image recognition, puzzles).

 BCS602 (Machine Learning)

o Early AI relied on expert systems that converted expert knowledge into rules (e.g., MYCIN for
medical diagnosis), but these lacked true intelligence.

Machine Learning Approach:

o Shifts from rule-based systems to data-driven models.

o Uses input data to develop models that can predict unknown data.

Concepts in Machine Learning:

Model: A mathematical or logical representation of patterns in data.

o Forms of models:

1. Mathematical equations.

2. Relational diagrams (e.g., trees, graphs).

3. Logical rules (if/else).

Clusters (groupings).

Pattern vs. Model:

o Pattern: Local, applies to certain attributes.

o Model: Global, fits the entire dataset.

Example: A model predicting whether an email is spam or not is derived automatically from
data.

Tom Mitchell's Definition of Machine Learning:

“A computer program learns from experience E, with respect to task T, and performance measure
P, if its performance on T measured by P improves with experience E.”

Example:

o Task (T): Object detection in images.

o Experience (E): Training on thousands of labeled images.

o Performance (P): Metrics like precision and recall measure detection accuracy.

Steps in Machine Learning:

1. Data Collection: Gathering raw data from various sources.

2. Abstraction: Forming abstract concepts from data (e.g., recognizing an elephant’s features).

3. Generalization:

o Converts abstractions into actionable intelligence.

 BCS602 (Machine Learning)

o Involves ranking concepts, inferencing, and forming heuristics (rules of thumb).

4. Evaluation:

o Tests the thoroughness of models.

o Identifies and corrects errors to improve system performance.

Machine Learning as Human Experience:

Computers mimic human experience through data.

Human Analogy:

o Humans learn by rote, observation, teachers, books, and trial-and-error.

o When faced with new problems, humans apply past experiences and heuristics.

Machine Analogy:

o Machines "learn" from data, generalize patterns, and form rules (heuristics) to solve tasks.

Role of Data in Machine Learning:

Quality of Data: Directly affects the accuracy of the learning system.

Statistical Learning Function: Maps input (x) to output (y) as y=f(x).

Goal: Learn the function f from data to accurately predict outcomes for new inputs.

Limitations of Heuristics:

Heuristics are educated guesses and may fail occasionally.

Course corrections are made through evaluation and improved formulations.

Machine Learning in Relation to Other Fields

Machine Learning and Artificial Intelligence (AI):

 BCS602 (Machine Learning)

Machine learning is a sub-branch of AI, which aims to develop intelligent agents (e.g., robots,
humans, or autonomous systems).

AI originally focused on logic and reasoning but evolved into data-driven systems for finding
relationships in data.

Machine learning focuses on extracting patterns for prediction, while deep learning, a sub-
branch of machine learning, uses neural networks modeled on human neurons.

Machine Learning and Data Science:

Data science is an umbrella term encompassing multiple fields, including machine learning.

Machine learning starts with data and focuses on analyzing patterns for prediction. Some Key
fields under data science include:
 BCS602 (Machine Learning)

Big Data: Focuses on large datasets with the following characteristics:

 Volume: Huge amounts of data (e.g., from Facebook, YouTube).

 Variety: Data in different forms (e.g., images, videos).

 Velocity: Speed at which data is generated and processed.

Data Mining: Unearths hidden patterns in data (similar to machine learning but focuses on
pattern discovery).

Data Analytics: Extracts useful knowledge from raw data. Predictive data analytics is closely
related to machine learning.

Pattern Recognition:

An engineering field that uses machine learning algorithms for feature extraction, pattern
analysis, and classification.

Considered a specific application of machine learning.

Machine Learning and Statistics:

Statistics:

Focuses on hypotheses, experiments, and rigorous mathematical models to find relationships in

data.

Requires a strong theoretical background and often involves complex equations and assumptions.

Machine Learning:

Less assumption-heavy compared to statistics and automates the learning process. Often viewed
as an evolved version of "old statistics."

Difference: Statistics aims to validate hypotheses, while machine learning focuses on predictions
and automation.

Some Key Takeaways:

Machine learning bridges AI, data science, statistics, and pattern recognition. Deep learning, big
data, and data analytics significantly influence its development.

Machine learning simplifies traditional statistical processes while enhancing predictive power.

Types of Machine Learning

 BCS602 (Machine Learning)

Machine learning involves learning through the interaction of a program with its environment.

The types of learning are categorized based on the data used and the supervision provided during
the learning process.

Labelled and Unlabelled Data Labelled Data:

 BCS602 (Machine Learning)

Data with target attributes (labels) is used for supervised learning.

For example, the Iris dataset includes features (e.g., sepal length, petal width) with a target label (class of
Iris flower).

Unlabelled Data:

Data without target attributes is used for unsupervised learning.

The algorithm identifies patterns or groups without explicit supervision.

Supervised Learning

Involves a teacher or supervisor providing labelled data for training and testing.

Two Key Methods: Classification:

Predicts discrete labels or classes.

Example: Classifying images as cats or dogs.

Algorithms:

 Decision Trees

 Random Forest

 Support Vector Machines (SVM)

 Naïve Bayes

 Artificial Neural Networks (e.g., CNNs for deep learning)

Classification models:
 BCS602 (Machine Learning)

Generative Models: Focus on data generation (e.g., probabilistic models).

Discriminative Models: Focus on classification only.

Regression:

Predicts continuous variables, such as prices or sales.

 BCS602 (Machine Learning)

Example: Predicting product sales over weeks using linear regression.

Algorithm:

Linear Regression: Fits a line (e.g., y=0.66x+0.54y = 0.66x + 0.54y=0.66x+0.54) to predict outcomes
based on independent variables.

Unsupervised Learning

Self-instruction-based learning with no supervisor or labelled data. Algorithms observe examples,

recognize patterns, and group similar objects.

Methods:

Cluster Analysis:

Groups objects into disjoint clusters based on attributes.

Example:

 Image segmentation to detect abnormal growth in medical images.

 Clustering gene signatures in a database.

Algorithms:

 k-means algorithm

 Hierarchical algorithms

Dimensionality Reduction:

 Simplifies data while retaining essential features.

 BCS602 (Machine Learning)

 Semi-Supervised Learning

 Combines labelled and unlabelled data, leveraging small labelled datasets and larger unlabelled
datasets.

 Useful when labelling data is expensive or time-consuming.

 Differences Between Classification and Regression

Dimensionality Reduction

Dimensionality reduction is a type of unsupervised learning where high-dimensional data is transformed

into a lower-dimensional space while retaining the critical variance or patterns in the dataset.

The goal is to simplify the dataset by reducing the number of features without losing its essential
information.

Differences between Supervised and Unsupervised Learning

Applications:

Reducing computational complexity for machine learning models

 Visualizing high-dimensional data in 2D or 3D (e.g., t-SNE for visualizing clusters).

 Preprocessing data to remove noise or redundant features.

Examples of Dimensionality Reduction Algorithms:

Principal Component Analysis (PCA): Projects data into components that explain the maximum
variance.

Linear Discriminant Analysis (LDA): Finds the linear combinations of features that best separate
classes.

t. Distributed Stochastic Neighbor Embedding (t-SNE): Focuses on visualizing high-

dimensional data in 2D or 3D.
 BCS602 (Machine Learning)

Autoencoders: Neural networks that compress and reconstruct data.

Semi-Supervised Learning

Semi-supervised learning operates in scenarios where a large portion of the dataset is unlabelled, and only
a small portion is labelled.

Since labelling data is expensive and time-consuming, semi-supervised algorithms make use of unlabelled
data by assigning pseudo-labels (predicted labels).

Process:

Unlabelled data is assigned pseudo-labels based on model predictions.

The pseudo-labelled data is combined with the labelled data to train the model.

Applications:

 Image recognition (with limited labelled samples).

 Natural language processing for text classification.

 Medical diagnosis where labelling data is costly.

Reinforcement Learning

Reinforcement learning is inspired by how humans learn through interaction and feedback.

An agent interacts with the environment, perceives its state, takes actions, and receives rewards or
penalties.

The agent aims to maximize cumulative rewards by learning the best actions to take over time.

Key Concepts:

Agent: The decision-maker (e.g., a robot, software, or a program).

 BCS602 (Machine Learning)

Environment: The space where the agent interacts.

Actions: Choices available to the agent (e.g., move left, right, up, down in a grid game).

Reward: Feedback signal for actions (positive or negative).

Policy: The strategy the agent uses to determine actions.

Goal: Achieve the highest cumulative reward over time.

Example - Grid Game:

In the grid game:

 Gray tiles represent danger.

 Black tiles are blocks.

 Diagonal-striped tile is the goal.

The agent learns to navigate from the bottom-left grid to the goal by exploring various paths and gaining
rewards for successful actions or penalties for poor decisions.

Characteristics of Reinforcement Learning:

No labelled data: The agent learns through interaction instead of relying on labelled datasets.

Sequential decision-making: The agent must take multiple steps to achieve the goal.

Reward-driven learning: Positive rewards reinforce desirable behavior, while negative rewards
discourage it.

Challenges of Machine Learning

Machine learning, while powerful, faces several challenges that stem from its reliance on data quality,
computational resources, and algorithmic efficiency.

Below are the key challenges, explained in detail:

Problems with Ill-posed Problems

Machine learning performs well on well-posed problems, where specifications are complete, clear, and
well-defined.

However, ill-posed problems—where there is insufficient information or multiple valid solutions—are a

significant challenge.
 BCS602 (Machine Learning)

o For instance, given a dataset where y = X1 X X2 , it’s equally plausible that y = X1 ÷ X2

or y = X1 + X2 . Without additional data or constraints, it’s impossible to determine the
correct function.

o Solution: Ill-posed problems require more comprehensive data, better- defined objectives, or
external constraints to improve their solvability.

Quality of Data

Machine learning systems heavily rely on data quality.

Challenges with data include:

o Insufficient Data: Models require large datasets to learn effectively.

o Incomplete Data: Missing values can reduce model accuracy.

Incorrect Data: Erroneous or noisy data introduces biases and errors.

o Imbalanced Data: Datasets where certain classes are underrepresented lead to poor performance
for those classes.

Solution: High-quality preprocessing, data augmentation, and imputation techniques can address
these issues.

High Computational Power Requirements

The growth of Big Data and complex machine learning tasks (e.g., deep learning) demands
significant computational resources:

Hardware Requirements: Systems need GPUs, TPUs, or distributed computing for faster
processing.

Time Complexity: Training sophisticated models can take hours, days, or even weeks.

Solution: Leveraging cloud computing platforms, distributed training, and optimized frameworks
(e.g., TensorFlow, PyTorch) can mitigate computational bottlenecks.

Algorithmic Complexity
 BCS602 (Machine Learning)

The complexity of machine learning algorithms presents challenges in:

Selection: Identifying the best algorithm for a specific task.

Implementation: Understanding and applying advanced algorithms.

Optimization: Designing and fine-tuning algorithms to maximize efficiency and accuracy.

Evaluation: Comparing algorithm performance to ensure optimal model selection.

Solution: Continuous research, education, and development of automated machine learning

(AutoML) tools simplify the design and selection process.

Bias/Variance Tradeoff

Bias and variance are sources of error in machine learning models:

Bias: Occurs when the model oversimplifies the problem, leading to underfitting.

Variance: Occurs when the model overfits the training data, failing to generalize to new data.

Striking the right balance between bias and variance—called the bias-variance tradeoff—is a
persistent challenge.

Solution:

Use cross-validation techniques to assess model performance.

Regularization methods (e.g., L1, L2 regularization) can help reduce overfitting.

Ensemble methods like bagging (e.g., Random Forests) and boosting (e.g., XGBoost) improve
generalization.

Interpretability and Explainability

Many machine learning models, particularly deep learning, are considered black boxes, making it
difficult to explain their predictions or decisions.

Solution:

Use simpler, interpretable models where feasible.

Employ explainability tools like SHAP (SHapley Additive exPlanations) or LIME (Local
Interpretable Model-agnostic Explanations).

Ethical Concerns and Bias in Data

Biases present in training data can lead to biased models, perpetuating unfair decisions (e.g., in hiring,
lending, or law enforcement).

Solution: Implement fairness-aware machine learning techniques, conduct audits, and use diverse
datasets.

Scalability
 BCS602 (Machine Learning)

Scaling machine learning models to work with massive datasets across distributed systems is a
technical challenge.

Solution: Use scalable tools and frameworks like Apache Spark, Dask, or cloud-based machine
learning services.

Machine Learning Process (CRISP-DM)

The CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology provides a
structured framework for building data mining and machine learning solutions.

It is widely adopted because of its versatility across industries. Below are the six steps of the CRISP-
DM process:

Understanding the Business

Objective: Grasp the business goals and define the problem statement.

Activities:

o Identify the organization's objectives and challenges.

o Formulate a clear problem statement for the machine learning solution.

o Choose a data mining or machine learning algorithm that aligns with the business goals.

Understanding the Data

Objective: Explore and understand the data available for solving the problem.
 BCS602 (Machine Learning)

Activities:

o Collect and analyze data from different sources.

o Study the data's characteristics, such as trends, anomalies, and distributions.

o Formulate hypotheses based on the data and match patterns to the hypotheses.

Data Preparation

Objective: Clean and preprocess the raw data for analysis.

Activities:

o Handle missing values, duplicates, and irrelevant data.

o Normalize or scale the data as required.

o Prepare a finalized dataset suitable for the machine learning task.

Importance: Data quality directly impacts the performance of the machine learning model.

For instance, missing values can degrade model accuracy.

Modeling

Objective: Apply machine learning algorithms to extract patterns or build models.

Activities:

o Select appropriate machine learning algorithms (e.g., classification, regression, clustering).

o Train the model on the prepared dataset.

o Tune hyperparameters to optimize model performance.

Evaluation

Objective: Assess the performance of the model using statistical and visualization techniques.

Activities:

o Measure accuracy, precision, recall, F1-score, and other metrics.

o Compare results with baseline models or domain benchmarks.

o Ensure the model aligns with business objectives.

Challenges: Tasks like email classification may require expert domain knowledge for proper
evaluation.

Deployment

Objective: Deploy the model to provide actionable insights or improve processes.

Activities:
 BCS602 (Machine Learning)

o Integrate the model into the business workflow or system.

o Monitor its performance in real-world scenarios.

o Update the model as new data becomes available.

Applications of Machine Learning

Machine learning is transforming various industries and is increasingly present in everyday life.

Below are some key applications:

Sentiment Analysis

Use Case: Natural Language Processing (NLP) techniques analyze text to determine sentiments (e.g.,
happy, sad, angry).

Example: Movie and product reviews can be analyzed to assign ratings (e.g., stars) automatically.

Recommendation Systems

Use Case: Personalizes user experiences by suggesting items based on preferences. Example:

 Amazon: Recommends related books or products.

 Netflix: Suggests shows or movies based on viewing history.

Voice Assistants

 Use Case: Use speech recognition and natural language understanding to perform tasks.

 Examples: Amazon Alexa, Apple Siri, Google Assistant, Microsoft Cortana. Navigation
Systems

 Use Case: Optimize routes and reduce travel time using machine learning-based algorithms.

 Examples: Google Maps, Uber.

 BCS602 (Machine Learning)

Big
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)
BCS602 (Machine Learning)