{

"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"from numpy.testing import assert_allclose"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part I. Construct a Householder reflection of a vector."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Given a vector $\\mathbf{x}$, and a plane with a normal vector $\\mathbf{u}$,
the Householder transformation reflects $\\mathbf{x}$ about the plane.\n",
"\n",
"The matrix of the Householder transformation is\n",
"\n",
"$$\n",
"\\mathbf{H} = \\mathbf{1} - 2 \\mathbf{u} \\mathbf{u}^T\n",
"$$\n",
"\n",
"Given two equal-length vectors $\\mathbf{x}$ and $\\mathbf{y}$, a rotation
which brings $\\mathbf{x}$ to $\\mathbf{y}$ is a Householder transform with\n",
"\n",
"$$\n",
"\\mathbf{u} = \\frac{\\mathbf{x} - \\mathbf{y}}{\\left|\\mathbf{x} -
\\mathbf{y}\\right|}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Write a function which rotates a given vector, $\\mathbf{x} = (x_1, \\dots,
x_n)$ into $\\mathbf{y} = (\\left|\\mathbf{x}\\right|, 0, \\dots, 0)^T$ using a
Householder transformation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
 "outputs": [],
"source": [
"def householder(vec):\n",
" \"\"\"Construct a Householder reflection to zero out 2nd and further
components of a vector.\n",
"\n",
" Parameters\n",
" ----------\n",
" vec : array-like of floats, shape (n,)\n",
" Input vector\n",
" \n",
" Returns\n",
" -------\n",
" outvec : array of floats, shape (n,)\n",
" Transformed vector, with ``outvec[1:]==0`` and ``|outvec| == |vec|
``\n",
" H : array of floats, shape (n, n)\n",
" Orthogonal matrix of the Householder reflection\n",
" \"\"\"\n",
" vec = np.asarray(vec, dtype=float)\n",
" if vec.ndim != 1:\n",
" raise ValueError(\"vec.ndim = %s, expected 1\" % vec.ndim)\n",
" \n",
" # ... ENTER YOUR CODE HERE ..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Test your function using tests below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Test I.1 (10% of the total grade).\n",
"\n",
"v = np.array([1, 2, 3])\n",
"v1, h = householder(v)\n",
"\n",
"assert_allclose(np.dot(h, v1), v)\n",
"assert_allclose(np.dot(h, v), v1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Test I.2 (10% of the total grade).\n",
 "\n",
"rndm = np.random.RandomState(1234)\n",
"\n",
"vec = rndm.uniform(size=7)\n",
"v1, h = householder(vec)\n",
"\n",
"assert_allclose(np.dot(h, v1), vec)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part II. Compute the $\\mathrm{QR}$ decomposition of a matrix.\n",
"\n",
"Given a rectangular $m\\times n$ matrix $\\mathbf{A}$, construct a Householder
reflector matrix $\\mathbf{H}_1$ which transforms the first column of $\\mathbf{A}$
(and call the result $\\mathbf{A}^{(1)}$)\n",
"\n",
"$$\n",
"\\mathbf{H}_1 \\mathbf{A} =%\n",
"\\begin{pmatrix}\n",
"\\times & \\times & \\times & \\dots & \\times \\\\\n",
"0 & \\times & \\times & \\dots & \\times \\\\\n",
"0 & \\times & \\times & \\dots & \\times \\\\\n",
"&& \\dots&& \\\\\n",
"0 & \\times & \\times & \\dots & \\times \\\\\n",
"\\end{pmatrix}%\n",
"\\equiv \\mathbf{A}^{(1)}\\;.\n",
"$$\n",
"\n",
"Now consider the lower-right submatrix of $\\mathbf{A}^{(1)}$, and construct a
Householder reflector which annihilates the second column of $\\mathbf{A}$:\n",
"\n",
"$$\n",
"\\mathbf{H}_2 \\mathbf{A}^{(1)} =%\n",
"\\begin{pmatrix}\n",
"\\times & \\times & \\times & \\dots & \\times \\\\\n",
"0 & \\times & \\times & \\dots & \\times \\\\\n",
"0 & 0 & \\times & \\dots & \\times \\\\\n",
"&& \\dots&& \\\\\n",
"0 & 0 & \\times & \\dots & \\times \\\\\n",
"\\end{pmatrix}%\n",
"\\equiv \\mathbf{A}^{(2)} \\;.\n",
"$$\n",
"\n",
"Repeating the process $n-1$ times, we obtain\n",
"\n",
"$$\n",
"\\mathbf{H}_{n-1} \\cdots \\mathbf{H}_2 \\mathbf{H}_1 \\mathbf{A} =
\\mathbf{R} \\;,\n",
"$$\n",
"\n",
"with $\\mathbf{R}$ an upper triangular matrix. Since each $\\mathbf{H}_k$ is
orthogonal, so is their product. The inverse of an orthogonal matrix is orthogonal.
Hence the process generates the $\\mathrm{QR}$ decomposition of $\\mathbf{A}$. "
]
},
{
 "cell_type": "markdown",
"metadata": {},
"source": [
"Write a function, which receives a recangular matrix, $A$, and returns the Q
and R factors of the $QR$ factorization of $A$."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def qr_decomp(a):\n",
" \"\"\"Compute the QR decomposition of a matrix.\n",
" \n",
" Parameters\n",
" ----------\n",
" a : ndarray, shape(m, n)\n",
" The input matrix\n",
" \n",
" Returns\n",
" -------\n",
" q : ndarray, shape(m, m)\n",
" The orthogonal matrix\n",
" r : ndarray, shape(m, n)\n",
" The upper triangular matrix\n",
" \n",
" Examples\n",
" --------\n",
" >>> a = np.random.random(size=(3, 5))\n",
" >>> q, r = qr_decomp(a)\n",
" >>> np.assert_allclose(np.dot(q, r), a)\n",
" \n",
" \"\"\"\n",
" a1 = np.array(a, copy=True, dtype=float)\n",
" m, n = a1.shape\n",
" \n",
" # ... ENTER YOUR CODE HERE ..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Might want to turn this on for prettier printing: zeros instead of `1e-16`
etc\n",
"\n",
"np.set_printoptions(suppress=True)"
]
},
{
"cell_type": "code",
 "execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Test II.1 (20% of the total grade)\n",
"\n",
"rndm = np.random.RandomState(1234)\n",
"a = rndm.uniform(size=(5, 3))\n",
"q, r = qr_decomp(a)\n",
"\n",
"# test that Q is indeed orthogonal\n",
"assert_allclose(np.dot(q, q.T), np.eye(5), atol=1e-10)\n",
"\n",
"# test the decomposition itself\n",
"assert_allclose(np.dot(q, r), a)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now compare your decompositions to the library function (which actually wraps
the corresponding LAPACK functions)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from scipy.linalg import qr\n",
"qq, rr = qr(a)\n",
"\n",
"assert_allclose(np.dot(qq, rr), a)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if your `q` and `r` agree with `qq` and `rr`. Explain."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Enter your explanation here* (10% of the total grade, peer-graded)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
 "# Part III. Avoid forming Householder matrices explicitly."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note the special structure of the Householder matrices: A reflector
$\\mathbf{H}$ is completely specified by a reflection vector $\\mathbf{u}$. Also
note that the computational cost of applying a reflector to a matrix strongly
depends on the order of operations:\n",
"\n",
"$$\n",
"\\left( \\mathbf{u} \\mathbf{u}^T \\right) \\mathbf{A} \\qquad \\textrm{is }
O(m^2 n)\\;,\n",
"$$\n",
"while\n",
"$$\n",
"\\mathbf{u} \\left( \\mathbf{u}^T \\mathbf{A} \\right) \\qquad \\textrm{is }
O(mn)\n",
"$$\n",
"\n",
"Thus, it seems to make sense to *avoid* forming the $\\mathbf{H}$ matrices.
Instead, one stores the reflection vectors, $\\mathbf{u}$, themselves, and provides
a way of multiplying an arbitrary matrix by $\\mathbf{Q}^T$, e.g., as a standalone
function (or a class)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Write a function which constructs the `QR` decomposition of a matrix *without
ever forming the* $\\mathbf{H}$ matrices, and returns the $\\mathbf{R}$ matrix and
reflection vectors. \n",
"\n",
"Write a second function, which uses reflection vectors to multiply a matrix
with $\\mathbf{Q}^T$. Make sure to include enough comments for a marker to follow
your implementation, and add tests. \n",
"\n",
"(Peer-graded, 40% of the total grade)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
 "language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}