Scribd
Upload
37 views

Lecture 6,7-Linear Regression

This document summarizes linear regression with one variable. It introduces the linear regression model, cost function, gradient descent algorithm, and how gradient descent is applied to learn the parameters for linear regression. Specifically, it discusses representing the model, choosing parameters to minimize cost, intuitions behind the cost function and gradient descent, and how the gradient descent algorithm is used to simultaneously update the parameters theta-0 and theta-1 to reduce cost until reaching the minimum.

Uploaded by

Waseem Shahzad
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
37 views

Lecture 6,7-Linear Regression

This document summarizes linear regression with one variable. It introduces the linear regression model, cost function, gradient descent algorithm, and how gradient descent is applied to learn the parameters for linear regression. Specifically, it discusses representing the model, choosing parameters to minimize cost, intuitions behind the cost function and gradient descent, and how the gradient descent algorithm is used to simultaneously update the parameters theta-0 and theta-1 to reduce cost until reaching the minimum.

Uploaded by

Waseem Shahzad
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 47

Linear regression

with one variable


Model
representation
500000
Housing Prices
400000

300000

Price 200000
(in 1000s 100000
of dollars) 0
500 1000 1500 2000 2500 3000
Size (feet2)
Supervised Learning Regression Problem
Given the “right answer” for Predict real-valued output
each example in the data.
Training set of Size in feet2 (x) Price ($) in 1000's (y)
housing prices 2104 460
(Portland, OR) 1416 232
1534 315
852 178
… …
Notation:
m = Number of training examples
x’s = “input” variable / features
y’s = “output” variable / “target” variable
Training Set How do we represent h ?

Learning Algorithm

Size of h Estimated
house price

Linear regression with one variable.


Univariate linear regression.
Linear regression
with one variable
Cost function
Size in feet2 (x) Price ($) in 1000's (y)
Training Set 2104 460
1416 232
1534 315
852 178
… …

Hypothesis:
‘s: Parameters
How to choose ‘s ?
3 3 3

2 2 2

1 1 1

0 0 0
0 1 2 3 0 1 2 3 0 1 2 3
y

Idea: Choose so that


is close to
for our training examples
Linear regression
with one variable
Cost function
intuition I
Simplified
Hypothesis:

Parameters:

Cost Function:

Goal:
(for fixed , this is a function of x) (function of the parameter )

3 3

2 2
y
1 1

0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
(for fixed , this is a function of x) (function of the parameter )

3 3

2 2
y
1 1

0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
(for fixed , this is a function of x) (function of the parameter )

3 3

2 2
y
1 1

0 0
0 1 2 3 -0.5 0 0.5 1 1.5 2 2.5
x
Linear regression
with one variable
Cost function
intuition II
Hypothesis:

Parameters:

Cost Function:

Goal:
(for fixed , this is a function of x) (function of the parameters )

500000

400000
Price ($) 300000
in 1000’s
200000

100000

0
500 1000 1500 2000 2500 3000
Size in feet2 (x)
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
Linear regression
with one variable

Gradient
descent
Have some function
Want

Outline:
• Start with some
• Keep changing to reduce
until we hopefully end up at a minimum
J(0,1)

1
0
J(0,1)

1
0
Gradient descent algorithm

Correct: Simultaneous update Incorrect:


Linear regression
with one variable
Gradient descent
intuition
Gradient descent algorithm
If α is too small, gradient descent
can be slow.

If α is too large, gradient descent


can overshoot the minimum. It may
fail to converge, or even diverge.
at local optima

Current value of
Linear regression
with one variable
Gradient descent for
linear regression
Gradient descent algorithm Linear Regression Model
Gradient descent algorithm

update
and
simultaneously
J(0,1)

1
0
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
(for fixed , this is a function of x) (function of the parameters )
“Batch” Gradient Descent

“Batch”: Each step of gradient descent


uses all the training examples.

You might also like