Machine Learning
Types
Supervised methods types, hypothesis function, cost function, and
gradient descent
Hypothesis:
Parameters:
Cost Function:
Goal:
Linear
regression with
one variable
Cost
function
Machine Learning intuition I
Simplified
Hypothesis:
Parameters:
Cost Function:
Goal:
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y
1 1
0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y
1 1
0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
(for fixed , this is a function of x) (function of the parameter )
3 3
2 2
y
1 1
0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
Linear
regression with
one variable
Cost
function
Machine Learning intuition II
(for fixed , this is a function of x) (function of the parameters )
500000
400000
Price ($) 300000
in 1000’s
200000
100000
0
500 1000 1500 2000 2500 3000
Size in feet (x)
2
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
Linear
regression with
one variable
Gradient
descent
Machine Learning
Have some function
Want
Outline:
• Start with some
• Keep changing to reduce
until we hopefully end up at a minimum
J(0,1)
1
0
J(0,1)
1
0
Gradient descent algorithm
Correct: Simultaneous update Incorrect:
Linear
regression with
one variable
Gradient
descent
intuition
Machine Learning
Gradient descent algorithm
If α is too small, gradient descent
can be slow.
If α is too large, gradient descent
can overshoot the minimum. It may
fail to converge, or even diverge.
Linear
regression with
one variable
Gradient
descent for
linear regression
Machine Learning
Gradient descent algorithm Linear Regression Model
Gradient descent algorithm
update
and
simultaneously
J(0,1)
1
0
Linear Regression
with multiple
variables
Multiple
features
Machine Learning
Multiple features (variables).
Size Price
(feet2) ($1000)
2104 460
1416 232
1534 315
852 178
… …
Multiple features (variables).
Size Number Price
(feet2) of Number Age of home ($1000)
bedrooms of floors (years)
2104 5 1 45 460
1416 3 2 40 232
1534 3 2 30 315
852 2 1 36 178
… … … … …
Notation:
= number of features
= input (features) of training example.
= value of feature in training example.
Hypothesis:
Previously:
For convenience of notation, define .
Multivariate linear regression.
Linear Regression
with multiple
variables
Gradient
descent for
multiple
variables
Machine Learning
Hypothesis:
Parameters:
Cost function:
Gradient descent:
Repeat
(simultaneously update for every )
New algorithm :
Gradient Descent
Repeat
Previously (n=1):
Repeat
(simultaneously update for
(simultaneously update )
Linear Regression
with multiple
variables
Gradient descent in
practice I: Feature
Scaling
Machine Learning
Feature Scaling
Idea: Make sure features are on a similar scale.
E.g. = size (0-2000 feet )
2 size (feet2)
= number of bedrooms (1-5)
number of bedrooms
Linear Regression with
multiple variables
Features and
polynomial regression
Machine Learning
Housing prices prediction
Andrew Ng
Polynomial regression
Price
(y)
Size (x)
Andrew Ng
Choice of features
Price
(y)
Size (x)
Andrew Ng
Linear Regression with
multiple variables
Normal equation
Machine Learning
Gradient Descent
Normal equation: Method to solve for
analytically.
Andrew Ng
Intuition: If 1D
(for every )
Solve for
Andrew Ng
Examples:
Size (feet2) Number of Number of Age of home Price ($1000)
bedrooms floors (years)
1 2104 5 1 45 460
1 1416 3 2 40 232
1 1534 3 2 30 315
1 852 2 1 36 178
Andrew Ng
is inverse of matrix .
Octave: pinv(X’*X)*X’*y
Andrew Ng
training examples, features.
Gradient Descent Normal Equation
• Need to choose . • No need to choose .
• Needs many iterations. • Don’t need to iterate.
• Works well even • Need to compute
when is large.
• Slow if is very large.
Andrew Ng
Regression Evaluation Metrics
1. Mean Absolute Error(MAE):
MAE =
2. Mean Squared Error(MSE)
MSE =
Andrew Ng
Regression Evaluation Metrics
3. Root Mean Squared Error(RMSE)
4. R Squared (R2)
RMSE =
= 1 - = 1- ,
Adjusted R2=
SSR = Residuals sum of squares
SST = Total sum of squares.
Andrew Ng
