Scribd
Upload
140 views

Machine Learniing

This document discusses linear regression with multiple variables. It covers gradient descent for multiple variables, feature scaling to standardize features, choosing a learning rate for gradient descent, adding polynomial features, and using the normal equation to solve for coefficients analytically. The normal equation can result in a non-invertible matrix with redundant or too many features, in which case some features may need to be removed or regularization used.

Uploaded by

anshul77
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
140 views

Machine Learniing

This document discusses linear regression with multiple variables. It covers gradient descent for multiple variables, feature scaling to standardize features, choosing a learning rate for gradient descent, adding polynomial features, and using the normal equation to solve for coefficients analytically. The normal equation can result in a non-invertible matrix with redundant or too many features, in which case some features may need to be removed or regularization used.

Uploaded by

anshul77
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 31

Llnear 8egresslon wlLh

muluple varlables
Muluple feaLures
Machlne Learnlng
Andrew ng
!"#$ &'$$(
)
*


+,"-$ &./000*


2104 460
1416 232
1334 313
832 178
. .
123453$ '$6(2,$7 &86,"693$7*:
Andrew ng
!"#$ &'$$(
)
*


;2<9$, ='
9$>,==<7

;2<9$, ='
?==,7

@A$ =' B=<$
&C$6,7*

+,"-$ &./000*


2104 3 1 43 460
1416 3 2 40 232
1334 3 2 30 313
832 2 1 36 178
. . . . .
123453$ '$6(2,$7 &86,"693$7*:
noLauon:
= number of feaLures
= lnpuL (feaLures) of Lralnlng example.
= value of feaLure ln Lralnlng example.
Andrew ng
PypoLhesls:
revlously:
Andrew ng
lor convenlence of noLauon, dene .
MuluvarlaLe llnear regresslon.
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL for
muluple varlables
Machlne Learnlng
Andrew ng
PypoLhesls:
CosL funcuon:
arameLers:
(slmulLaneously updaLe for every )
8epeaL
CradlenL descenL:
Andrew ng
(slmulLaneously updaLe )
D,6>"$E( F$7-$E(
8epeaL
revlously (n=1):
new algorlLhm :
8epeaL
(slmulLaneously updaLe for
)
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL ln
pracuce l: leaLure Scallng
Machlne Learnlng
Andrew ng
L.g. = slze (0-2000 feeL
2
)
= number of bedrooms (1-3)
G$6(2,$ !-63"EA
ldea: Make sure feaLures are on a slmllar scale.
slze (feeL
2
)
number of bedrooms
Andrew ng
G$6(2,$ !-63"EA
CeL every feaLure lnLo approxlmaLely a range.
Andrew ng
8eplace wlLh Lo make feaLures have approxlmaLely zero mean
(uo noL apply Lo ).
1$6E E=,<63"#64=E
L.g.
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL ln
pracuce ll: Learnlng raLe
Machlne Learnlng
Andrew ng
D,6>"$E( >$7-$E(
- uebugglng": Pow Lo make sure gradlenL
descenL ls worklng correcLly.
- Pow Lo choose learnlng raLe .
Andrew ng
Lxample auLomauc
convergence LesL:
ueclare convergence lf
decreases by less Lhan
ln one lLerauon.
0 100 200 300 400
no. of lLerauons
16H"EA 72,$ A,6>"$E( >$7-$E( "7 I=,H"EA -=,,$-(3C:
Andrew ng
16H"EA 72,$ A,6>"$E( >$7-$E( "7 I=,H"EA -=,,$-(3C:
CradlenL descenL noL worklng.
use smaller .
no. of lLerauons
no. of lLerauons no. of lLerauons
- lor sumclenLly small , should decrease on every lLerauon.
- 8uL lf ls Loo small, gradlenL descenL can be slow Lo converge.
Andrew ng
!2<<6,CJ
- lf ls Loo small: slow convergence.
- lf ls Loo large: may noL decrease on
every lLerauon, may noL converge.
1o choose , Lry
Llnear 8egresslon wlLh
muluple varlables
leaLures and
polynomlal regresslon
Machlne Learnlng
Andrew ng
K=27"EA 5,"-$7 5,$>"-4=E
Andrew ng
+=3CE=<"63 ,$A,$77"=E
rlce
(y)
Slze (x)
Andrew ng
LB="-$ =' '$6(2,$7
rlce
(y)
Slze (x)
Llnear 8egresslon wlLh
muluple varlables
normal equauon
Machlne Learnlng
Andrew ng
CradlenL uescenL
normal equauon: MeLhod Lo solve for
analyucally.
Andrew ng
lnLuluon: lf 1u
Solve for
(for every )
Andrew ng
!"#$ &'$$(
)
*


;2<9$, ='
9$>,==<7

;2<9$, ='
?==,7

@A$ =' B=<$
&C$6,7*

+,"-$ &./000*


1 2104 3 1 43 460
1 1416 3 2 40 232
1 1334 3 2 30 313
1 832 2 1 36 178
!"#$ &'$$(
)
*


;2<9$, ='
9$>,==<7

;2<9$, ='
?==,7

@A$ =' B=<$
&C$6,7*

+,"-$ &./000*


2104 3 1 43 460
1416 3 2 40 232
1334 3 2 30 313
832 2 1 36 178
Lxamples:
Andrew ng
$M6<53$7 N '$6(2,$7:
L.g. lf
Andrew ng
ls lnverse of maLrlx .
CcLave: pinv(X*X)*X*y
Andrew ng
(,6"E"EA $M6<53$7O '$6(2,$7:
CradlenL uescenL normal Lquauon
no need Lo choose .
uon'L need Lo lLeraLe.
need Lo choose .
needs many lLerauons.
Works well even
when ls large.
need Lo compuLe
Slow lf ls very large.
Llnear 8egresslon wlLh
muluple varlables
normal equauon
and non-lnverublllLy
(opuonal)
Machlne Learnlng
Andrew ng
normal equauon
- WhaL lf ls non-lnveruble? (slngular/
degeneraLe)
- CcLave: pinv(X*X)*X*y
Andrew ng
WhaL lf ls non-lnveruble?
8edundanL feaLures (llnearly dependenL).
L.g. slze ln feeL
2

slze ln m
2

1oo many feaLures (e.g. ).
- ueleLe some feaLures, or use regularlzauon.

You might also like