AI Module 1 (Theory Questions)
What is Machine Learning (ML)?
Machine Learning is a branch of Artificial Intelligence that
enables computers to learn patterns from data and make
decisions or predictions without being explicitly programmed.
What are the main types of Machine Learning systems?
The main types are:
Supervised Learning
Unsupervised Learning
Semi-supervised Learning
Reinforcement Learning
Differentiate between supervised and unsupervised learning.
In supervised learning, the model is trained on labeled data
(input-output pairs). In unsupervised learning, the model works
on data without labels to find hidden patterns or structures.
List some common challenges in Machine Learning.
Insufficient or poor-quality data
Overfitting and underfitting
Data imbalance
Model interpretability
High computational cost
What is overfitting? How can it be avoided?
Overfitting occurs when a model learns noise or random
fluctuations in training data, reducing performance on new
data. It can be avoided by using regularization, cross-validation,
or collecting more data.
Explain a typical supervised learning workflow.
Collect labeled data
Split into training and testing sets
Train the model on the training set
Validate and tune parameters
Test performance on the unseen testing set
Give an example of a regression model.
Linear regression predicts a continuous output variable, such as
predicting house prices based on area and location features.
Differentiate between regression and classification problems.
Regression predicts continuous numerical values, whereas
classification predicts discrete categorical outcomes like “spam”
or “not spam.”
Explain the concept of logistic regression.
Logistic regression is a classification algorithm that predicts the
probability of a categorical dependent variable using a logistic
(sigmoid) function to map outputs between 0 and 1.
Mention some evaluation metrics used in classification.
Common metrics include accuracy, precision, recall, F1-score,
and confusion matrix.
What is K-means clustering?
K-means clustering is an unsupervised learning algorithm that
partitions data into K clusters by minimizing the sum of squared
distances between data points and their cluster centroids.
Describe the steps in the K-means clustering algorithm.
Initialize K cluster centroids randomly
Assign each data point to the nearest centroid
Recompute centroids as the mean of assigned points
Repeat until centroids stabilize
What are some limitations of K-means clustering?
Requires pre-specifying K
Sensitive to initial centroid placement
Assumes spherical clusters of similar sizes
What is an Artificial Neural Network (ANN)?
An ANN is a computational model inspired by the human brain,
composed of interconnected nodes (neurons) that process
input to generate outputs through weighted connections and
activation functions.
Define a perceptron.
A perceptron is the simplest form of a neural network
consisting of a single neuron with an input layer and a weighted
sum followed by an activation function to produce an output.
State the Universal Approximation Theorem.
The theorem states that a feedforward neural network with at
least one hidden layer containing a finite number of neurons
can approximate any continuous function on a closed and
bounded subset of real numbers.
What is a Multi-Layer Perceptron (MLP)?
An MLP is a feedforward neural network with one or more
hidden layers that can learn complex nonlinear relationships
between inputs and outputs.
How does a Deep Neural Network differ from an MLP?
A Deep Neural Network (DNN) is an extension of MLP with
many hidden layers, enabling high-level feature extraction and
hierarchical learning.
How can MLP be used for regression problems?
In regression, the MLP’s output layer uses a linear activation
function to predict continuous numerical outputs, optimizing a
loss function like Mean Squared Error (MSE).
How does MLP handle classification problems?
For classification, the MLP uses a softmax or sigmoid activation
in the output layer to produce probabilities for different classes,
optimizing loss functions such as cross-entropy loss.
Describe the K-means clustering algorithm and discuss its
limitations.
Answer: K-means starts by choosing K initial centroids, assigns
each data point to the nearest centroid, recomputes centroids
as the mean of assigned points, and repeats assignment and
update steps until convergence. Its limitations include the need
to pre-specify K, sensitivity to initial centroid positions, and
difficulty handling non-spherical clusters or clusters with
different densities and sizes.
Explain the architecture and learning process of a Multi-Layer
Perceptron.
Answer: An MLP consists of an input layer, one or more hidden
layers with nonlinear activation functions (e.g., ReLU, sigmoid),
and an output layer whose activation depends on the task
(linear for regression, softmax/sigmoid for classification).
During learning, the network performs forward propagation to
compute outputs and a loss value, and then uses
backpropagation with gradient-based optimization (e.g., SGD,
Adam) to update weights and biases to minimize the loss over
training data.
How can an MLP be used for regression and classification
problems?
Answer: For regression, the MLP typically uses a linear
activation in the output layer and optimizes a loss such as Mean
Squared Error to predict continuous targets. For classification,
the MLP uses sigmoid (binary) or softmax (multiclass) activation
in the output layer and optimizes cross-entropy loss to estimate
class probabilities, which are then converted to predicted labels
using a decision rule like argmax.
Discuss common challenges in Machine Learning and how they
relate to model generalization.
Answer: Key challenges include overfitting, where the model
memorizes training data and fails to generalize; underfitting,
where the model is too simple to capture patterns; data quality
issues such as noise and missing values; and class imbalance,
which biases the model towards frequent classes. Addressing
these through regularization, careful model selection,
appropriate evaluation metrics, and robust data preprocessing
improves generalization performance on unseen data.
Explain the difference between AI, Machine Learning, and Deep
Learning with suitable examples.
Answer: Artificial Intelligence is the broader field concerned
with creating systems that exhibit intelligent behavior, such as
reasoning, planning, and problem-solving. Machine Learning is
a subfield of AI that focuses on learning patterns from data to
make predictions or decisions, for example, using regression to
predict house prices. Deep Learning is a subfield of Machine
Learning that uses deep neural networks with many layers to
automatically learn complex representations, such as using
convolutional neural networks for image recognition.
Describe in detail the main categories of Machine Learning:
supervised, unsupervised, semi-supervised, and reinforcement
learning.
Answer: In supervised learning, models learn from labeled
examples where each input has an associated target, and the
objective is to learn a mapping for prediction. In unsupervised
learning, models discover patterns, clusters, or structures in
unlabeled data, such as grouping customers by purchasing
behavior. Semi-supervised learning uses a small amount of
labeled data combined with a large amount of unlabeled data
to improve performance. Reinforcement learning involves an
agent that interacts with an environment, receiving rewards or
penalties, and learns a policy that maximizes long-term
cumulative reward.
Discuss in detail the common challenges in Machine Learning
and categorize them as data-related, model-related, and
deployment-related issues.
Answer: Data-related challenges include insufficient data, noisy
or inconsistent labels, missing values, and dataset shift between
training and deployment environments. Model-related
challenges include overfitting, underfitting, model selection,
hyperparameter tuning, and interpretability of complex models
like deep networks. Deployment-related issues involve
scalability, latency constraints, model drift over time,
monitoring performance in production, and ethical
considerations such as bias and fairness.
Explain the concepts of bias, variance, and the bias–variance
trade-off in the context of Machine Learning models.
Answer: Bias refers to systematic error introduced by using
overly simple assumptions in the model, which can lead to
underfitting if the model cannot capture the underlying pattern.
Variance refers to the model’s sensitivity to small fluctuations in
the training data, where highly complex models may overfit by
learning noise. The bias–variance trade-off describes the need
to balance model complexity: increasing complexity can reduce
bias but increase variance, and the goal is to choose a model
that minimizes the total generalization error.
Describe different types of data preprocessing steps required
before training Machine Learning models, giving examples for
each.
Answer: Data preprocessing often includes handling missing
values using imputation or deletion, and treating outliers
through capping or robust transformations. Feature scaling
methods such as normalization or standardization ensure that
features with different ranges do not dominate distance-based
or gradient-based algorithms. Encoding categorical variables
using techniques like one-hot encoding or ordinal encoding,
along with feature selection or dimensionality reduction,
further prepares the dataset to improve learning efficiency and
model performance.
Explain linear regression in detail, including the model
equation, assumptions, and typical applications.
Answer: Linear regression models the relationship between
input features and a continuous output as a linear combination
of the features plus an error term. Common assumptions
include linearity, independence of errors, homoscedasticity
(constant variance of errors), and approximate normality of
residuals. It is widely used in applications such as demand
forecasting, cost estimation, and trend analysis where
interpretability of coefficients and simplicity are important.
Discuss the differences between simple linear regression,
multiple linear regression, and polynomial regression with
examples.
Answer: Simple linear regression uses one independent variable
to predict a continuous output with a straight-line relationship.
Multiple linear regression extends this by using several
independent variables, enabling the model to capture the
influence of multiple factors, such as predicting house prices
using area, location, and number of rooms. Polynomial
regression allows nonlinear relationships by including higher-
order terms of the features, effectively fitting curves while still
being linear in the parameters.
Explain logistic regression as a classification algorithm, including
its decision boundary and how probabilities are converted into
class labels.
Answer: Logistic regression uses a linear combination of input
features as input to a logistic (sigmoid) function, which outputs
a probability between 0 and 1 for the positive class. The
decision boundary is defined where the predicted probability
equals a threshold, typically 0.5, corresponding to a linear
boundary in feature space for binary classification. Class labels
are obtained by assigning samples with probability above the
threshold to the positive class and those below to the negative
class, although the threshold can be adjusted based on
application requirements.
Describe in detail the key evaluation metrics for classification,
and explain when accuracy can be misleading.
Answer: Classification metrics include accuracy, which measures
the proportion of correctly classified samples, and precision and
recall, which focus on performance on the positive class. The
F1-score combines precision and recall into a single harmonic
mean, while the confusion matrix provides a detailed
breakdown of true positives, true negatives, false positives, and
false negatives. Accuracy can be misleading in imbalanced
datasets where one class dominates, because a model that
always predicts the majority class may achieve high accuracy
but perform poorly on the minority class of real interest.
Explain the concepts of training set, validation set, and test set,
and discuss why splitting data properly is important.
Answer: The training set is used to fit the model’s parameters
and learn patterns from the data. The validation set is used to
tune hyperparameters, compare models, and prevent
overfitting by assessing performance during development. The
test set is held out until the end to provide an unbiased
estimate of generalization performance, and improper splitting
or reusing test data for tuning can lead to overly optimistic and
unreliable performance estimates.
Describe the K-means clustering algorithm in detail and
illustrate its working with a conceptual numerical example
(without computation).
Answer: K-means starts with an initial choice of K cluster
centers, either randomly or using a heuristic, then iteratively
alternates between assigning each data point to its nearest
center and recomputing centers as the mean of assigned points.
Conceptually, in a two-dimensional dataset, points close
together will be grouped into the same cluster, and the
centroids move until assignments no longer change
significantly. The algorithm aims to minimize the total within-
cluster sum of squared distances, leading to compact, roughly
spherical clusters.
Discuss the factors that influence the choice of K in K-means
and mention common methods used to select an appropriate
number of clusters.
Answer: The choice of K affects both model complexity and
interpretability; too few clusters can merge distinct groups,
while too many clusters can lead to over-partitioning and noise
fitting. Common methods for choosing K include the elbow
method, which examines how the within-cluster sum of squares
decreases as K increases, and the silhouette score, which
evaluates how well-separated and compact the clusters are.
Practical considerations such as domain knowledge and the
intended use of clusters also guide the final choice of K.
Compare K-means clustering with hierarchical clustering and
briefly indicate when K-means may be unsuitable.
Answer: K-means assumes convex, roughly spherical clusters
and works best when clusters have similar sizes and densities,
providing a flat partition of the data into K groups. Hierarchical
clustering builds a tree-like structure of nested clusters, either
merging small clusters into larger ones (agglomerative) or
splitting large clusters into smaller ones (divisive), and does not
require pre-specifying K. K-means may be unsuitable when
clusters are non-spherical, have significantly different sizes, or
when there is substantial noise and outliers, where density-
based or hierarchical methods may perform better.
Explain the structure and mathematical model of a perceptron,
and discuss the concept of linear separability.
Answer: A perceptron takes several inputs, multiplies each by a
weight, adds a bias term, and passes the resulting weighted
sum through a step or other activation function to produce a
binary output. The decision boundary corresponds to a linear
hyperplane in the input space, and the perceptron can correctly
classify data only if the classes are linearly separable. Linear
separability means that a single straight line (in two
dimensions) or hyperplane (in higher dimensions) can separate
the data points of different classes without error.
Describe the perceptron learning algorithm and its convergence
property for linearly separable data.
Answer: The perceptron learning algorithm initializes weights
and bias, then iteratively updates them whenever a training
example is misclassified by adjusting weights in the direction of
the correct class. If the data is linearly separable, the algorithm
is guaranteed to converge in a finite number of steps to a set of
weights that perfectly separates the classes. However, for non-
linearly separable data, the algorithm may never converge and
can oscillate among different weight configurations.
State the Universal Approximation Theorem and discuss its
importance in the context of neural networks and function
approximation.
Answer: The Universal Approximation Theorem states that a
feedforward neural network with at least one hidden layer
containing a finite number of neurons and a suitable nonlinear
activation function can approximate any continuous function on
a compact domain to arbitrary accuracy. This result is important
because it provides theoretical justification that even relatively
simple network architectures have enough representational
power to model complex relationships. However, the theorem
does not guarantee efficient training, nor does it specify how
many neurons are required in practice.
Explain the architecture of a Multi-Layer Perceptron (MLP) and
describe the role of activation functions in hidden and output
layers.
Answer: An MLP consists of an input layer that receives feature
vectors, one or more hidden layers with multiple neurons, and
an output layer tailored to the task, such as a single neuron for
regression or multiple neurons for multiclass classification.
Activation functions in hidden layers, such as ReLU, sigmoid, or
tanh, introduce nonlinearity, enabling the network to learn
complex, non-linear mappings between inputs and outputs. The
output layer activation is chosen according to the problem:
linear for regression, sigmoid for binary classification, and
softmax for multiclass classification.
Discuss how backpropagation and gradient-based optimization
are used to train MLPs, highlighting the main steps involved.
Answer: Training an MLP begins with forward propagation,
where inputs are passed through the layers to compute the
output and the loss function is evaluated based on the
difference between predictions and targets. Backpropagation
then computes gradients of the loss with respect to weights
and biases by applying the chain rule layer by layer from the
output backward to the input. Gradient-based optimization
algorithms, such as stochastic gradient descent or Adam, use
these gradients to update parameters iteratively, gradually
reducing the loss and improving the model’s performance on
the training data.
Differentiate between a shallow neural network and a deep
neural network, and discuss the advantages and challenges of
using deep architectures.
Answer: A shallow neural network has one or very few hidden
layers, whereas a deep neural network has multiple hidden
layers stacked between the input and output. Deep
architectures can learn hierarchical feature representations,
where early layers capture simple patterns and deeper layers
capture more abstract concepts, leading to state-of-the-art
performance in tasks like image and speech recognition.
However, they introduce challenges such as higher
computational cost, risk of overfitting, vanishing or exploding
gradients, and the need for large labeled datasets and careful
regularization.
Explain how a Multi-Layer Perceptron can be used to solve both
regression and classification problems, highlighting differences
in network design and training objectives.
Answer: For regression problems, the MLP usually has a linear
activation in the output neuron(s) and is trained using a
regression loss such as Mean Squared Error or Mean Absolute
Error to predict continuous values. For classification, the output
layer uses sigmoid (for binary) or softmax (for multiclass)
activation to produce probabilities, and training minimizes a
classification loss such as binary or categorical cross-entropy.
Although the underlying architecture can be similar, the choice
of output layer, loss function, and evaluation metrics differs
according to whether the task is predicting continuous
quantities or categorical labels.
