[Exercise 2] 多元线性回归

上一节我们实现了二元线性回归。在这个练习中，会使用梯度下降和normal equations实现多元线性回归.同时也会检查损失函数，收敛梯度和学习率的关系。参考Exercise: Multivariate Linear Regression我们要预测的是1650，且有3个bedrooms的房子的价格。

Data

先看看数据集，训练集是Porland,Oregon地区的价格作为$y^{(i)}$,输入$x^{(i)}$是living area和bedrooms 大小。一共有m=47个样列。

实验基础

梯度下降

• 假设函数依然如下：
  
• 梯度更新
  

• 损失函数：
  

• 损失函数的向量形式：
  
  其中：

normal equations

使用这个公式不需要任何特征缩放,你会得到一个确切的解决方案在一个计算:没有像在梯度下降循环,直到收敛

程序实验和结果

alpha =

    0.0100    0.0300    0.1000    0.3000    1.0000    1.3000


theta_grad_descent =

   1.0e+05 *

    3.4041
    1.1063
   -0.0665


price_grad_descent =

   2.9308e+05

normal equations

% normal equations
x = load('ex3x.dat');
y = load('ex3y.dat');

x = [ones(size(x,1),1), x]

theta_norequ = inv((x'*x))*x'*y

price_norequ = theta_norequ'*[1 1650 3]'

梯度计算：

%Multivariate Linear Regression
x = load('ex3x.dat');
y = load('ex3y.dat');

% add the x0=1 intercept term into your x matrix
x = [ones(size(x,1),1), x];

sigma = std(x); %standard deviations
mu = mean(x); % means

x(:,2) = (x(:,2) - mu(2))./ sigma(2);
x(:,3) = (x(:,3) - mu(3))./ sigma(3);

% gradient descent
figure
alpha = [0.01,0.03,0.1,0.3,1,1.3]
J =zeros(50, 1);
itera_num = 100; 
sample_num = size(x,1);
plotstyle = {'b','r','g','k','b--', 'r--'};

theta_grad_descent = zeros(size(x(1,:)));

for alpha_i = 1:length(alpha)
    theta = zeros(size(x(1,:)))';
    Jtheta = zeros(itera_num,1);
    for i = 1:itera_num
        Jtheta(i) = (0.5/sample_num).*(x*theta - y)'*(x*theta - y);
        grad = (1/sample_num).*x'*(x*theta - y);
        theta = theta - alpha(alpha_i).*grad;
    end
    plot(0:49, Jtheta(1:50),char(plotstyle(alpha_i)), 'LineWidth', 2)
    hold on
    if (1==alpha(alpha_i))
       theta_grad_descent = theta
    end

end

legend('0.01', '0.03', '0.1', '0.3', '1', '1.3');

% now plot J
% technically, the first J starts at the zero-eth iteration

xlabel('Number of iterations')
ylabel('Cost J')

% predict price
price_grad_descent = theta_grad_descent'*[1 (1650-mu(2))/sigma(2) (3-mu(3))/sigma(3)]'

参考资料

1. http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=DeepLearning&doc=exercises/ex3/ex3.html
2. http://www.cnblogs.com/tornadomeet/archive/2013/03/15/2962116.html