Applied projects for an

introductory linear algebra class

Anna Zemlyanova
 This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 Interna-
tional License. To view a copy of this license, visit http://creativecommons.org/licenses/
by-nc/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042,
USA.

2
Contents
About this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
A brief introduction to Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Advice on good programming style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Project overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Project 1: Basic operations with matrices in Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Project 2: Matrix operations and image manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Project 3: Matrix multiplication, inversion, and photo filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Project 4: Solving linear systems in Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Project 5: Systems of linear equations and college football team ranking (with an example of
the Big 12). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Project 6: Convolution, inner product, and image processing revisited . . . . . . . . . . . . . . . . . . . . . . 40
Project 7: Norms, angles, and your movie choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Project 8: Interpolation, extrapolation, and climate change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Project 9: Orthogonal matrices and 3D graphics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
Project 10: Discrete dynamical systems, linear transformations of the plane, and the Chaos
Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Project 11: Projections, eigenvectors, Principal Component Analysis, and face recognition algo-
rithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Project 12: Matrix eigenvalues and the Google’s PageRank algorithm. . . . . . . . . . . . . . . . . . . . . . .74
Project 13: Social networks, clustering, and eigenvalue problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Project 14: Singular Value Decomposition and image compression . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

3
About this book
This book is based on the Applied Matrix Theory curriculum which the author taught at Kansas
State University. Applied Matrix Theory is an introductory linear algebra undergraduate class
geared primarily towards engineering students. Unlike a proof-based class that is typically of-
fered to mathematics majors, the focus here is to introduce real life applications at the same
time as teaching the standard topics of linear algebra such as matrix operations, linear systems,
inverses, vector spaces, determinants, and eigenproblems. While teaching the class, the author
of this book struggled with finding realistic applications of the theoretical material which can
be reasonably completed during a class period. The search for real life applications has led the
author to write this book which hopefully fills the gap between linear algebra theory and appli-
cations. Most of the projects in this book can be done with minimal knowledge of programming.
It is author’s hope that the projects will encourage the exploration of data using simple concepts
from linear algebra. The projects are written in Matlab in order to harness the abundance of
built in functions and computational power of Matlab. The author attempted to incorporate
real data from various open sources into these projects. There are two labs which are dedicated
to exploring operations with matrices and to solving linear systems in Matlab (the projects 1
and 4). There are also two labs which use more advanced material typically not covered in an
introductory linear algebra class, such as singular value decomposition and normal equations
(the projects 8 and 14).
The code in this book has been developed using the 2015a version of Matlab and tested on
Matlab 2019a. Depending on the version of Matlab on your computer the syntax and the output
might differ slightly. The projects in this book can be adapted to use with GNU Octave as well.
The supplementary files necessary to complete some of the labs are available from the author’s
webpage at http://www.math.ksu.edu/~azem.
A variety of printed and Internet sources inspired this book. These sources are acknowledged
in the Reference section as well as the sources of data for the projects. This book has been
developed under the Open/Alternative Textbook Initiative grant from Kansas State University,
the generous support of which is gratefully acknowledged here. The author is immensely grateful
to her colleagues, Nathan Albin and Pietro Poggi-Corradini, for proofreading the book and
for the corrections. The author is grateful to Jeffrey Sheldon for proofreading the book and
suggesting style corrections.

4
A brief introduction to Matlab
MATLAB is a numerical computing software application and a high level programming lan-
guage developed by MathWorks, Inc. Initially, Matlab stood for “matrix laboratory” and was
developed specifically to perform numerical linear algebra operations. Currently Matlab has a
vast library of built in mathematical functions which allow users to perform complex computa-
tions without having to program numerical methods themselves. Because of this Matlab is the
software of choice for many scientific and engineering applications.
In this book we will use Matlab to illustrate common linear algebra concepts and to perform
computations on problems with real life applications. Before we go any further, let us review
some basic commands and principles of using Matlab. For more information, please see Matlab
Help or any Matlab manual of your choice.

The Matlab desktop environment

Matlab’s desktop environment typically consists of several windows. By default, you are likely
to see the command window with a prompt. There is also usually the “Current Folder” window
which shows the directory on the computer you are working with and the “Workspace” window
with any variables which have been initialized (if you did not define any variables yet this window
will be empty). In some cases you can also see the “Command History” window which shows
the commands which have been executed recently. The default configuration can be changed
by clicking downward arrow in the upper right corner of each of these windows or by using the
switch windows button located in the upper right corner of the main Matlab window. Matlab can
be used by typing commands in a command prompt or by executing text files with commands.
These files are called Matlab scripts and have the extension “.m”. Matlab comes with a built
in editor for editing M-files. This editor can also be one of the windows you see in your Matlab
environment when you start Matlab.

The command line

The easiest way to get started with Matlab is by executing commands in the interactive command
window. The command prompt can be seen in the main window of the Matlab environment:
>>

A Matlab command can be entered in this prompt and Matlab will produce an immediate
result. For instance, entering
>> sin(pi/3)
will generate the output
ans =
0.8660
To suppress Matlab output you can put a semicolon at the end of your command. For instance,
the command
>> sin(pi/3);
produces no output. However, observe that the last output which has not been assigned to any
variable is stored in the Matlab variable ans. This variable has now changed its value to sin π/3
rounded up to the machine precision.

5
 By default Matlab uses the short format for output on the screen which is given by the
Matlab command format short. This command generates an output for real numbers with up
to 4 decimal digits. The output format can be changed by executing the command
format long
The long format produces output for real numbers with up to 15 decimal digits.
If you want to know more about a particular command, you can always type
help <name of the command>
or
doc <name of the command>
This will open Matlab Help on the command you are interested in and also produce various
examples of Matlab’s use of this command. You can also open Matlab help directly by pressing
the corresponding button in the Matlab environment. This button looks like a question mark
and is generally located in the upper right corner of the Matlab “Home” menu.
Another useful tip to remember is that pressing “Up” arrow button on your keyboard while
in the command line will bring up previous commands you have executed. You can scroll through
the commands and select any of them to run again. This is especially useful if you have been
typing lengthy expressions in the command line and need to correct misprints or simply need to
run the command again.
Finally, sometimes it is necessary to interrupt Matlab computations. This might happen, for
instance, if Matlab is stuck in an infinite loop or a computation simply takes too long to finish.
You can interrupt the execution of Matlab code by pressing “Ctrl-C” while in the main Matlab
window.

Matlab scripts
While simple computations can be handled by typing commands directly into the command
prompt, it is advisable to organize more complex computations in a form of Matlab scripts or
M-files. You can create a new script file by going to the menu “Home” of the main Matlab
window and clicking on the “New Script” button. This will open a new empty script file in the
Matlab editor.
It is a good idea to keep your M-files well-documented by utilizing the comment option in
Matlab. You can create a comment in a script file by starting the code line with the percentage
symbol %. While interpreting the script, Matlab will ignore everything in the line after the %
symbol.
You can also create code cells within the script. A new code cell can be started by using the
double percentage symbol %%. You can also write a title of the cell after the %% sign. Matlab will
treat everything after the %% sign as a comment. The creation of code cells allows you to break
your code into smaller executable pieces and run those pieces individually. To do so you can use
the “Run and Advance” button at the top of the Matlab editor window. Generally speaking,
you want to split the code into code cells in some logical way. For instance, each code cell can
be dedicated to performing a particular function in your code.
Finally, you can publish your code using the “Publish” section of the menu of the Matlab
editor. By default, code is published in the HTML format. Publishing will create a document
which involves a table of contents, the code itself, and all the plots and output which have been
produced by your code.

6
Variables, vectors, matrices
Variables are used to store and manipulate the values obtained during a Matlab session. Matlab
uses the following standard types of variables: real which can be double and single precision
(double or single), several integer types (int8, int16, int32, uint8, uint16, uint32, uint64),
boolean (logical) and character (char).
By default all the numbers in Matlab are treated as the type double which uses 64 bits to
store a real number.
Each variable needs to have an identifier or a name. There are some general considerations
for variable names in Matlab:

• The name of a variable must start with a letter. After the first character the name may
contain letters, numbers, and the underscore character.

• Variable names in Matlab are case-sensitive. This means that Price and price are treated
as different variables.

• Certain words (such as reserved words or keywords) cannot be used as variable names.

• Names of built-in functions can, but generally should not, be used as variable names.
Matlab has several different ways to define matrices (two-dimensional arrays) and vectors
(one-dimensional arrays). For instance, we can define arrays explicitly by typing in all the values:
>> vec=[1 2 3 4]
vec =
1 2 3 4
>> mat=[1 2 3; 4 5 6]
mat =
1 2 3
4 5 6
In the example above a 1 × 4 vector vec and a 2 × 3 matrix mat are defined. Another way to
define vectors and matrices is by using the colon operator “:”. For instance, the variables vec
and mat from the example above can be also defined by commands
>> vec=1:4;
>> mat=[1:3;4:6];
We will explore other matrix operations further in this book. Matlab also has an ability to work
with multi-dimensional arrays which will be useful for us later when manipulating color images.
Matlab has a set of operators to perform standard algebraic operations with variables:

Operator Meaning
+, − addition, subtraction, binary
− negation, unitary
∗, \ multiplication, division, binary
^ power
() parentheses (change of the order of operations)

Keep in mind the priority in which these operators are executed: the parentheses () take
the highest priority, followed by the power operator ^, unitary negation −, multiplication and
division ∗, \, and, finally, addition and subtraction +, −.

7
Matlab functions
Matlab has a large library of built-in functions which makes it a convenient environment for
many scientific and engineering computations. The full list of available functions can be found
in Matlab Help and on the mathworks.com webpage. You can also look into the “Help” menu
of the Matlab main window (shown as a question mark in the upper right corner). The list of
elementary functions can be viewed by executing the command
>> help elfun
from the command line.
Matlab also allows user to create their own user-defined functions which can be saved in
separate M-files and can be later called from other scripts. The syntax of such functions in its
simplest form looks like this:
function [OutputVariables] = FunctionName(InputVariables)
% Description of the function
<Main body of the function>
end

Control statements
There are several standard control statements in Matlab which change the order in which the
commands of a script are executed or whether they are executed at all.
The control statements can be roughly divided into condition statements and loop state-
ments. The first group of control statements we will look at is the conditional statements. The
conditional statements allow one to perform different computations or procedures depending on
whether the condition (which is given by a boolean expression) is true or false. In the simplest
form, the conditional statement looks like this:
if <condition>
<statements>
else
<statements>
end
In the above construction, else statements can be skipped. Additionally, more options can be
incorporated by using the elseif clause. A construction of this type has the form:
if <condition1>
<statements1>
elseif <condition2>
<statements2>
elseif <conditions3>
<statements3>
...
else
<statements>
end
The switch statement can also be used in place of if...elseif statements which use many
elseif clauses. Conditions in each of the conditional statements represent logical (boolean)
expressions which can be either true or false.
The second group of control statements consists of loop statements. A simple for loop looks
like this

8
 for LoopVariable = Range
<statements>
end
Range here consists of all the values through which the LoopVariable will iterate. For
instance, in the following code
for i=1:3
fprintf(’%d\n’,i)
end
the loop variable i iterates through the integers 1, 2, and 3, and the code itself prints the integers
from 1 to 3 in a column format.
Another loop statement in Matlab is the while loop. The general form of a while statement
looks like this:
while <condition>
<statements>
end
The <statements> part of the code is executed as long as the conditional statement given by
<condition> remains true.

Logical statements
Expressions which are either true or false are called boolean or logical expressions. The logical
operators and relational operators in Matlab are given in the table below:

Operator Meaning
> (<) greater than (less than)
>= (<=) greater or equal (less or equal)
== logical equality
∼= logical inequality
|| or
&& and
∼ not
xor exclusive or

The difference between the logical equality == and the assignment operator = is that the
assignment operator assigns the value on the right side of the operator to the variable on the
left side of the operator. At the same time, the logical equality operator checks whether the two
values on the right and the left sides of the operator are equal.
When we compare two expressions using relational operators the result is a boolean which
can be true or false. For instance,
>> 3>2
ans =
1

The variable ans here is of logical type.

9
Input and output functions
The simplest way to create a user defined variables in Matlab is provided by the function input.
The input function displays a prompt string on the screen and awaits for user input. For
instance,
>> side=input(’Enter the side of the square: ’)
Enter the side of the square: 3
side =
3

The code above asks to input a number (the side of the square) and stores it in the variable
side. It is generally a good idea to give the user an explanation of what is expected from them.
The string ’Enter the side of the square: ’ above provides such an explanation.
Formatted output can be achieved in Matlab by using the fprintf function. For instance,
>> fprintf(’The area of the square is equal to %d\n’,side^2)
The area of the square is equal to 9
The expression %d here is a placeholder for the variable written after the string which, in this
case, is the value side2 . The symbol \n indicates that any output after this command should
start with a new line.
More information about the functions input and fprintf can be obtained by calling Matlab
Help with the help input or help fprintf commands.

Plotting graphs
A standard way to display information in the form of a two-dimensional graph in Matlab is
provided by the plot function. In the simplest form, the function plot looks like this:
>> plot(x_vector,y_vector,’linestyle’)
Here, the variables x vector and y vector are the vectors which contain, correspondingly, the
x- and y-coordinates of the points on the graph. The ’linestyle’ is a string variable which
contains the parameters for determining the type of line used for the graph. For instance, ’k-o’
means that the graph will be drawn with black (’k’) solid (’-’) line and each data point will
be shown with an empty circle (’o’).
It is possible to plot multiple graphs on the same figure at the same time:
>> plot(x1_vector,y1_vector,’linestyle1’,x2_vector,y2_vector,’linestyle2’, ...)
Alternatively, it is possible to add more graphs to the figure later by using the hold on,
hold off commands. If you don’t use the command hold on after the graph, calling the plot
function again will plot your graph in the previous figure window, erasing everything which was
on that figure before. If you would like a new plot to be shown in a new figure window without
erasing any of the previous plots, you can create a new figure window by using the figure
command. Finally, Matlab allows figures to be grouped together in the form of subfigures. This
can be achieved by using the subplot command.

Further reading
This introduction is designed to touch upon some of the basics of Matlab and is not meant to be
exhaustive. Some of the concepts discussed here will be applied in the sections to follow. There
are many other sources of additional information about Matlab, which include Matlab Help, the
mathworks.com website, and various Matlab manuals which can be purchased separately.

10
Advice on good programming style
Usually the goal of a code developer is not only to write working code but also to make the code
easily understandable to other people. Here are several fairly standard tips which the author
found useful on how to write clean, easily readable, code:

1. Choose meaningful names for your variables and functions. This will make your code
significantly easier to read. For instance, the meaning of the variables Height and Weight
is much clearer than non descriptive h and w. If the variable name consists of several
words, such as carspeed and priceperitem, then capitalizing each word (for instance,
CarSpeed and PricePerItem) or using underscores (car speed and price per item) can
increase the readability of your code.

2. If your code is long, break it into smaller pieces. Give each piece only one function.
In Matlab, you can create user-defined functions and code cells for this purpose. User
defined functions can be created by using the function keyword (type doc function in
the Matlab command line to learn more about it). Code cells can be created by entering
the double percentage sign %% into your code. Each code cell can then be run individually
by using the “Run and Advance” button in the Matlab editor.

3. Add comments to your code by using the single percentage sign %. Matlab will ignore
the rest of the line after the percentage sign. Explain the purpose of the commands used,
variables, and functions created in the comments. You will thank yourself for doing this
when you look at your code six months later. Anybody else trying to understand your
code will thank you too.

4. Don’t write dense code. Allow empty lines between the commands.

5. Use consistent indentation. This makes reading code with multiple cycles much easier.
Compare the two examples in the Fig. 1. Which one is easier to read?

Figure 1: An example of indentation usage.

6. Don’t make your lines too long. Split long commands and computations using triple dots
...

7. Clear code is usually better than clever code. Sometimes you can use a clever style of
programming to compound 5-10 lines of code into one single line. However, this code is
often much more difficult to understand.
You will discover many other tips and pointers on your own as you gain more experience
with programming.

11
Project overview
Each of the projects in this book is designed to test several concepts from linear algebra. The
goals of each project and linear algebra concepts used in it are summarized below:

Project 1: To introduce operations with matrices in Matlab such as matrix addition, scalar
multiplication, matrix multiplication, and matrix input functions in Matlab.

Project 2: To use the matrix operations introduced in the previous project to manipulate
images. Lightening/darkening, cropping, and changing the contrast of the images are discussed.

Project 3: To use matrix multiplication to manipulate image colors similarly to photo filters
for images.

Project 4: Introduction to solving linear systems in Matlab.

Project 5: To discuss some of the strategies for the ranking of sports teams with an example
of the Big 12 conference in college football.

Project 6: To apply convolution of two matrices to blurring/sharpening of images.

Project 7: To create a simple recommender system based on the use of norms and inner
products.

Project 8: To study basic interpolation techniques and the least squares method with
application to climate data.

Project 9: To use orthogonal matrices for the rotation of 3D objects.

Project 10: To apply matrix mappings in order to generate fractals with the help of Chaos
game.

Project 11: To apply eigenvalue problems and orthogonal projections in order to create a
simple face recognition algorithm.

Project 12: To introduce Google’s PageRank algorithm and look at the eigenproblems for
the ranking of webpages.

Project 13: Application of eignevalues and eigenvectors for the data clustering with the
example of Facebook network.

Project 14: Application of singular value decomposition for image compression and noise
reduction.

Sample questions to check students’ understanding and checkpoints for introduced variables
are highlighted in the text of the projects in yellow. The templates of the labs 9-14 are given in
the appendix.

12
Project 1: Basic operations with matrices in Matlab
Goals: To introduce matrix input functions and basic operations with matrices in Matlab.
The matrix operations to be studied include matrix addition and subtraction, scalar product,
matrix product and elementwise matrix product in Matlab, matrix concatenation, and selecting
submatrices.

To get started: Create a new Matlab script file and save it as lab01.m.

Matlab commands used: length, linspace, size, max, min, mean, sum,
randi (rand), ’, *, .*, eye, zeros, ones, diag, spdiags.

What you have to submit: The file lab01.m, which you will create during the lab session.

INTRODUCTION

It is assumed that the reader is familiar with basic matrix operations and their properties,
so the definitions and facts here are given mostly for the sake of a review.
We can think of a matrix as a table of numbers:
 
a11 a12 . . . a1j . . . a1n
 a21 a22 . . . a2j . . . a2n 
 
 · · · · · · ... · · · · · · · · · 
 
A=  
 ai1 ai2 . . . aij . . . ain 

 · · · · · · · · · · · · ... · · · 
 

am1 am2 . . . amj . . . amn

The matrix A above contains m rows and n columns. We say that A is an m × n (m-by-n)
matrix, and call m, n the dimensions of the matrix A. The matrix A above can also be compactly
written as A = (aij ), where aij is called (i, j) element of the matrix A.
Matlab (originally named “MATrix LABoratory”) has an extensive list of functions which
work with matrices. The operations discussed in this project consist of matrix addition/subtrac-
tion, multiplication of a matrix by a constant (scalar multiplication), transposition of a matrix,
selecting submatrices, and concatenating several smaller matrices into a larger one, matrix multi-
plication and componentwise matrix multiplication, and operations of finding maximal/minimal
element in the matrix, and finding the sum/mean in the row/column of the matrix.
Scalar multiplication of a matrix A by a constant c consists of multiplying every element of
the matrix by this constant:
cA = (caij ) .
Matrix addition and subtraction are possible if the matrices have the same dimensions and
is done componentwise:
A ± B = (aij ± bij ) .
If A is an m × n matrix then the transpose AT (A’ in Matlab) of the matrix A is the n × m
matrix obtained by flipping A along its diagonal entries. Thus every column of A becomes a
row of AT , and every row of A becomes a column of AT . For instance,
 
 T 1 4
1 2 3
= 2 5 .
4 5 6
3 6

13
 If n is a positive integer then the n × n identity matrix, denoted by In , is a matrix which has
1 in every diagonal entry (i, i) for 1 ≤ i ≤ n, and 0 in every other entry. There is one identity
matrix for every dimension n. For instance,
 
  1 0 0
1 0
I2 = I3 = 0 1 0
0 1
0 0 1
are the 2 × 2 and 3 × 3 identity matrices.
The identity matrices are special examples of diagonal matrices. A matrix A is diagonal if
A(i, j) is 0 whenever i 6= j (it is not required that aii 6= 0 for any i).
A matrix product C = AB of an m × n matrix A and an l × p matrix B exists if and only
if n = l. If this condition is satisfied, then C is an m × p matrix with the elements
n
X
cij = aik bkj .
k=1

Observe that matrix multiplication

  is non-commutative, AB 6= BA.
2 3  
4 0 2
For example if A =  1 0  and B = then the product AB is given by
3 −1 1
−1 −1
   
(2 · 4 + 3 · 3) (2 · 0 + 3 · −1) (2 · 2 + 3 · 1) 17 −3 7
AB =  (1 · 4 + 0 · 3) (1 · 0 + 0 · −1) (1 · 2 + 0 · 1)  =  4 0 2 .
(−1 · 4 + −1 · 3) (−1 · 0 + −1 · −1) (−1 · 2 + −1 · 1) −7 1 −3
 
  1 2 3
2 1
If C = and D = 0 1 2 then the product CD is not defined because the number
1 7
0 0 1
of columns of C does not equal the number of rows of D.

TASKS
1. Open the lab01.m Matlab script you have created and type in the following commands:
%% basic operations with matrices
A=[1 2 -10 4; 3 4 5 -6; 3 3 -2 5];
B=[3 3 4 2];

This will create a new cell and define two new variables A and B. The variable A stores
a 3 × 4 matrix, while the variable B contains a 1 × 4 row vector. Use the “Workspace”
window of the Matlab environment to view the variables A and B.
Variables: A, B

2. Type in the following commands:

lengthA = length(A)
lengthB = length(B)

Variables: lengthA, lengthb

Q1: What does the command length(A) do?

14
3. Add the vector B as the fourth row of the matrix A and save the result as a new matrix C
using the following code:
C=[A; B];

Variables: C

4. Create a new matrix D whose entries are located in the rows 2, 3, 4 and columns 3, 4 of the
matrix C:
D=C(2:4,3:4);

Variables: D

5. Create a transpose of the matrix D by using a quotation mark sign ’ in Matlab and save
the transposed matrix as the matrix E.
Variables: E

6. Check the size of the matrix E by using the Matlab size function and store the result of
this operation as two variables m, n.
Variables: m, n

7. Matlab allows for the creation of equally spaced vectors which can be useful in many
situations. For instance, run the following code:
EqualSpaced=0:pi/10:2*pi
EqualSpaced1=linspace(0,2*pi,21)

Take a look at the output produced by Matlab. Observe that both of these commands
created the vector with equally spaced entries ranging from 0 to 2π and the distances
between the elements equal to π/10.
Variables: EqualSpaced, EqualSpaced1

8. Create a new cell and find the maximal and minimal elements in each of the columns of
the matrix A. Store the result respectively in the row vectors named maxcolA and mincolA.
You can use the Matlab max and min functions:
%% Max, min, sum and mean
maxcolA=max(A)
mincolA=min(A)

Variables: maxcolA and mincolA

9. Now repeat the previous step but find the maximal and the minimal elements in each
row of the matrix A. Observe that there are at least two ways to approach this: you can
either use the max and min functions on the transpose of A or use these functions with
an additional argument 2 to show that you are interested in the maximum and minimum
along the second dimension (i.e. max(A,[],2)). Save the results as maxrowA and minrowA.
Finally find the maximal and minimal elements in the whole matrix A, and save the results
as maxA and minA.
Variables: maxrowA, minrowA, maxA and minA

15
10. Repeat the last two steps using the commands mean and sum instead of max and min. You
should create six new variables, corresponding to the column/row means/sums, and the
mean/sum of the elements of the entire matrix.
Variables: meancolA, meanrowA, sumcolA, sumrowA, meanA, sumA
Q2: What do the commands mean and sum do?

11. Create a new cell and use the command randi([-4 4],5,3) to create two matrices F and
G with 5 rows and 3 columns and random integer entries from −4 to 4:
%% Matrix addition/subtraction, and multiplication
F=randi([-4 4],5,3)
G=randi([-4 4],5,3)

Variables: F, G

12. Perform the operations 0.4*F, F+G, F-G, F.*G, storing the results in ScMultF, SumFG,
DiffFG, and ElProdFG respectively.
Variables: ScMultF, SumFG, DiffFG, ElProdFG
Q3: What does the last operation do?

13. Check the size of F and the size of A, storing the results in the variables sizeF and sizeA,
respectively.
Variables: sizeF, sizeA
Q4: Can we matrix-multiply F and A? Explain.

14. Compute H=F*A.

Variables: H
Q5: What are the dimensions of H?

15. Generate the identity matrix eye33 with 3 rows and 3 columns using the Matlab eye
command.
Variables: eye33

16. Run the commands:

zeros53=zeros(5,3);
ones42=ones(4,2);

Variables: zeros53, ones42

Q6: What do the functions zeros and ones do?

17. Generate the diagonal 3 × 3 matrix S with the diagonal elements {1, 2, 7} using the Matlab
diag function.
Variables: S
18. Now let us do the opposite: extract the diagonal elements from the matrix and save them
in a separate vector. The same function diag accomplishes that as well:
R=rand(6,6)
diagR=diag(R)

16
 This creates the 6 × 6 matrix R with random entries from the interval (0, 1), extracts
the diagonal entries from it, and saves them in the 1 × 6 vector diagR. Observe that the
function diag has other interesting functions. To learn more about it type help diag in
the command line.
Variables: R,diagR

19. Another function which allows to create diagonal matrices is spdiags. Technically, this
function creates a sparse matrix which is a matrix with a large number of zero elements.
These matrices are stored in a special form in Matlab with only non-zero elements are
stored, and operations with them are also done in a special way. To convert sparse form of
the matrix into the regular one, the command full can be used. Run the following code:
diag121=spdiags([-ones(10,1) 2*ones(10,1) -ones(10,1)],[-1 0 1],10,10);
full(diag121)

Variables: diag121
Q7: What does this code do?

17
Project 2: Matrix operations and image manipulation
Goals: To apply elementary matrix operations such as matrix addition, subtraction, and el-
ementwise multiplication to image manipulation. We will look at the lightening/darkening of
images, selecting subimages, changing contrast, rotating and flipping images.

To get started:

• Create a new matlab script and save it as lab02.m.

• Download the image file einstein.jpg and save it in your working directory 1 .

Matlab commands used: imread, imshow, uint8, double, flip, size, ’, zeros,
rot90, floor, ceil, round, fix.

What you have to submit: The file lab02.m, which you will create during the lab session.

INTRODUCTION

An image in a computer memory can be stored as a matrix with each element of the matrix
representing a pixel of the image and containing a number (or several numbers) which corre-
sponds to the color of this pixel. If the image is a color image, then each pixel is characterized
by three numbers corresponding to the intensities of Red, Green, and Blue (the so-called RGB
color system). If the image is a grayscale image, then only one number for the intensity of gray
is needed. The intensity of each color typically ranges from 0 (black) to 255 (white). This allows
for the representation of over 16 million colors in the RGB system which is more than a human
eye can distinguish. The images stored in this way are called bitmaps (there are also images
which do not use pixels, such as vector graphics, but we will not talk about those here).
If you have ever used an image editor before, you may know that there are several fairly
standard operations which you can perform with images. In this project we will replicate some
of them using operations on matrices. By the end of the project you will understand how
graphic editors process images to achieve different visual effects. As it turns out, many of these
operations are accomplished by manipulating the values of the pixel colors.

TASKS

1. To begin let us load the image into Matlab. This can be done by using the Matlab command
imread. This command allows Matlab to read graphic files with different extensions. The
output is an array with the dimensions equal to the dimensions of the image, and the
values corresponding to the colors of the pixels. Here we will work with a grayscale image,
so the elements in the matrix will be integers ranging from 0 to 255 in the Matlab integer
format uint8. Type in the following code to load the file “einstein.jpg” into Matlab:
%% Load a grayscale jpg file and represent the data as a matrix
ImJPG=imread(’einstein.jpg’);
1
The image is available at: https://commons.wikimedia.org/wiki/File:Albert_Einstein_Head.jpg and has
no known copyright restrictions. You can also use an alternative .jpg grayscale image of your choice. The answers
and the variable values will differ from what is shown here in that case.

18
 The array ImJPG is a two-dimensional array of the type uint8 which contains values from 0
to 255 corresponding to the color of each individual pixel in the image, where 0 corresponds
to black and 255 to white. You can visualize this array by going to the “Workspace” tab
of the main Matlab window and double clicking on ImJPG.
Variables: ImJPG
2. Use the size function to check the dimensions of the obtained array ImJPG:
[m,n]=size(ImJPG)

Variables: m, n
Q1: What are the dimensions of the image?

3. Check the type of the array ImJPG by using the command isinteger:
isinteger(ImJPG)

The output of the isinteger command is boolean, meaning it will produce either 1 (true)
or 0 (false).

4. Find the range of colors in the image by using max and min commands and save those
elements as maxImJPG and minImJPG. Observe that execution of this command depends on
the version of Matlab on your computer.
Variables: maxImJPG, minImJPG

5. Finally, display the image on the screen by using imshow:

imshow(ImJPG);

If you did everything correctly, you should see the image on the Fig. 2 displayed on your
screen in a separate window.

Figure 2: The original image

6. Now we are ready to work with this image. Let us look at some basic image operations
we can do using matrices. It is possible to crop the image by selecting a submatrix of the
matrix ImJPG. Selecting a submatrix can be done simply by naming the rows and columns
you want to keep from the initial matrix. For instance, the following commands
%% Select and display a part of the image, paste into a different image;
ImJPG_center = ImJPG(100:m-100, 100:n-70);
figure;
imshow(ImJPG_center);

19
 will create a new code cell, select the central part of the image leaving out 100 pixels from
the top and bottom, 100 pixels on the left and 70 pixels on the right, and display the result
in a new figure window.
Variables: ImJPG center
7. We can paste the selected part of the image into another image. To do this create a zero
matrix using the command
ImJPG_border=uint8(zeros(m,n));

Then paste the preselected matrix ImJPG center into matrix ImJPG border and display
the image:
ImJPG_border(100:m-100, 100:n-70)=ImJPG_center;
figure;
imshow(ImJPG_border);

Notice the conversion command uint8. It is necessary to use this command because by
default the array will be of the type double, and imshow command does not work correctly
with this type of array. To see this, try to remove uint8 and observe what happens when
you run the code above.
Variables: ImJPG border
8. The familiar operation of flipping the image vertically can be easily realized in Matlab by
using the matrix command flip:
%% Rotate and flip the image horizontally/vertically
ImJPG_vertflip=flip(ImJPG);

This command reverses the order of elements in each column of the matrix.
Variables: ImJPG vertflip

9. Transposing the matrix is achieved by adding an apostrophe ’ at the end of the matrix
name. Observe that transposing the image matrix is equivalent to rotating the image 90
degrees counterclockwise and flipping it horizontally. Try the following commands:
ImJPG_transpose=ImJPG’;
figure;
imshow(ImJPG_transpose);

Variables: ImJPG transpose

10. Flipping the image horizontally is a little more involved and requires the use of a matrix
transposition as well. For instance, this can be accomplished by using the command:
ImJPG_horflip=flip(ImJPG_transpose)’;

Display the resulting matrix and observe what happened to the initial image.
Variables: ImJPG horflip

11. Apply the command rot90 to the image using the following code:
ImJPG90=rot90(ImJPG)
figure;
imshow(ImJPG90)

20
 Variables: ImJPG90
Q3: What does the command rot90 do?
12. Try running the following commands:
%% Color inversion, lightening/darkening
ImJPG_inv = 255-ImJPG;
figure;
imshow(ImJPG_inv);

Display the resulting image in a new figure window. Observe also, that an array ImJPG
is subtracted here from the constant 255 which mathematically does not make sense.
However, Matlab treats the constant 255 as an array of the same size as ImJPG with all
the elements equal to 255.
Variables: ImJPG inv
Q4: Explain what happened to the image.
13. It is also easy to lighten or darken images using matrix addition. For instance, the following
code will create a darker image:
ImJPG_dark=ImJPG-50;

You can darken the image even more by changing the constant to a number larger than
50. Observe that the command above can technically make some of the elements of the
array to become negative. However, because the array type is uint8, those elements are
automatically rounded to zero.
Variables: ImJPG dark
Q5: How do you lighten the image?
14. Let us create Andy Warhol style art with the image provided. To do so we will arrange four
copies of the image into a 2 × 2 matrix. For the top left corner we will take the unaltered
image. For the top right corner we will darken the image by 50 shades of gray. For the
bottom left corner, lighten the image by 100 shades of gray, and finally, for the bottom
right corner, lighten the image by 50 shades of gray. Then we will arrange the images
together in one larger matrix using matrix concatenation. Finally, display the resulting
block matrix as a single image. If you did everything correctly, you should see an image
resembling the Fig. 3.
Variables: ImJPG Warhol
15. Matlab has several commands which allow one to round any number to the nearest integer
or a decimal fraction with a given number of digits after the decimal point. Those functions
include: floor which rounds the number towards negative infinity (to the smaller value),
ceil which rounds towards positive infinity (to the larger value), round which rounds
towards the nearest decimal or integer, and fix which rounds towards zero.
A naive way to obtain black and white conversion of the image can be accomplished by
making all the gray shades which are darker or equal to a medium gray (described by a
value 128) to appear as a complete black, and all the shades of gray which are lighter than
this medium gray to appear as white. This can be done, for instance, by using the code:
%% A naive conversion to black and white
ImJPG_bw=uint8(255*floor(ImJPG/128));
figure
imshow(ImJPG_bw);

21
 Figure 3: Andy Warhol style art with matrix concatenation

Observe that this conversion to black and white results in a loss of many details of the
image. There are possibilities to create black and white conversions without losing so
many details. Also notice the function uint8 used to convert the result back to the integer
format.
Variables: ImJPG bw

16. Now let us write code which reduces the number of shades in the image from 256 to 8 by
using round function. Save your array as ImJPG8 and display it in a separate window.
Variables: ImJPG8
17. The contrast in an image can be increased by changing the range of the possible shades
of gray in the image. Namely, we can increase the difference between the colors of the
image, for instance, by making the dark shades of gray to appear even darker. One of the
simplest ways to do this is to scalar multiply the matrix by some constant. Consider the
code:
ImJPG_HighContrast=uint8(1.25*ImJPG);

Display the image and observe the result. The contrast here can be manipulated by
increasing/decreasing the constant (we select 1.25 here). If we select a positive constant
greater than 1, then the contrast is increased; if we select a constant less than 1, then it
is decreased. Observe that this operation might lead to some of elements of the matrix to
become outside of 0 − 255 range and, hence, might lead to data loss.
Variables: ImJPG HighContrast

18. Finally, try another operation which is not linear but is still interesting. This operation is
called gamma-correction. Run the following code and observe the result:
%% Gamma correction
ImJPG_Gamma05=uint8((double(ImJPG)).^0.95);

22
 figure;
imshow(ImJPG_Gamma05);
ImJPG_Gamma15=uint8((double(ImJPG)).^1.05);
figure;
imshow(ImJPG_Gamma15);

Variables: ImJPG Gamma05, ImJPG Gamma15

23
Project 3: Matrix multiplication, inversion, and photo filters
Goals: We will use matrix multiplication in order to manipulate image color and achieve dif-
ferent effects.

To get started:
• Create a new Matlab script and save it as lab03.m.
• Download the file daylilies.jpg 2 and save it in your working directory. You can use an
alternative .jpg file of your choice. Your outputs will be different from shown here in that
case.

Matlab commands used: imread, imshow, size, figure, reshape, uint8, double,
for... end

What you have to submit: The file lab03.m which you will create during the lab session.

INTRODUCTION

Have you ever wondered how photo filters work? In this project we will learn how to manip-
ulate images in order to obtain different color effects. As we learned in the previous project, an
image can be represented by a matrix with the dimensions corresponding to the dimensions of
the image. Each of the elements of this matrix contains the number (or numbers) representing
the color of the corresponding pixel. If the image is a color image, then the color of the pixel is
represented by three numbers {R, G, B} (Red, Green and Blue) with each number ranging from
0 to 255 (RGB system). We will look at several different ways to manipulate the image colors
to obtain effects similar to those of Instagram and other graphics software.

TASKS

1. Start by loading the image into Matlab and displaying it on the screen by using the
familiar commands imshow and imread. Save the resulting array as ImJPG. You should
see something similar to the image on the Fig. 4.
Variables: ImJPG
2. Check the dimensions of the array ImJPG using the command
[m,n,l]=size(ImJPG)

Observe that Matlab will return three numbers. Since the image is colored, the resulting
array is three-dimensional. Among other things, Matlab allows operations on multidimen-
sional arrays. The first two dimensions of the array ImJPG correspond to the horizontal
and vertical dimensions (number of pixels) in the image, and the third dimension stores
three numbers corresponding to the values of Red, Green, and Blue for each pixel. These
three colors mixed together produce all other colors in the image. The values for Red,
Green and Blue can range from 0 to 255, allowing us to create 2563 = 16, 777, 216 colors.
In this project, we will be manipulating these three numbers to achieve various visual
effects (similar to the way photo filters work).
Variables: m,n,l
2
The copyright to this photo belongs to the author.

24
 Figure 4: The original image showing orange and yellow daylilies

3. First of all, let us look into amount of Red, Green, and Blue color in the image. To do
this, let us extract individual layers or color channels from the image array. To obtain the
red channel of color in the image, use the following command:
redChannel = ImJPG(:, :, 1);

Similarly, extract green and blue color channels and save them in the arrays greenChannel
and blueChannel. Display each array in a separate figure using figure and imshow
commands. Observe that individual color channels will be shown by Matlab as grayscale
images. That is due to the fact that we have taken “layers” off of the matrix ImJPG and
each of these layers individually looks like a two-dimensional array with numbers ranging
from 0 to 255 which is exactly the form in which grayscale images are represented in
Matlab. If you did everything correctly, you should see something similar to the Fig. 5.
The whiter the pixel appears on a particular color channel, the larger amount of that color
is contained in that pixel. On the other hand, the darker the area is, the less corresponding
color is in that part of the image. The file daylilies.jpg appears to contain mostly green
and red (surprise, surprise!).
Variables: redChannel, greenChannel, blueChannel

Figure 5: Different color channels

4. Let us convert the image into a grayscale image by using the following filter matrix:
 
1/3 1/3 1/3
GrayM atrix =  1/3 1/3 1/3 
1/3 1/3 1/3

Multiplication by this matrix allows to compute an average of red, green, and blue in each
pixel. Use the following code:

25
 GrayMatrix=[1/3 1/3 1/3; 1/3 1/3 1/3; 1/3 1/3 1/3];
for i=1:m
for j=1:n
PixelColor=reshape(double(ImJPG(i,j,:)),3,1);
ImJPG_Gray(i,j,:)=uint8(GrayMatrix*PixelColor);
end;
end;
figure;
imshow(ImJPG_Gray)

This code produces an array ImJPG Gray in which the individual values of Red, Green,
and Blue have been replaced by the averages of these three colors. Observe the use of the
commands reshape, uint8, double. For every pixel of the image, first, the color attributes
(RGB) are extracted from the matrix ImJPG, then these color attributes are treated as a
vector with three components [R, G, B], and finally, the new color attributes are obtained
by multiplying the vector [R, G, B] by the filter matrix GrayM atrix on the left. The result
is saved as color attributes of the corresponding pixel in the array ImJPG Gray. All the
pixels of the array ImJPG Gray have equal number of Red, Green, and Blue; this produces
different shades of gray color. Observe that there are different ways to create a grayscale
conversion, and this is just one of them.
Variables: ImJPG Gray
Q1: Explain the purpose of uint8 and double commands in the code above.

5. Modify the code above to produce a sepia conversion of the image. Instead of GrayMatrix
use the following filter matrix, and reproduce the code above with this matrix:
 
0.393 0.769 0.189
SepiaM atrix =  0.349 0.686 0.168 
0.272 0.534 0.131

Save the result in the ImJPG Sepia array and display this array using imshow command.
Variables: ImJPG Sepia

6. Next, consider the filter matrix

 
1 0 0
RedM atrix =  0 0 0 
0 0 0

Modify the code again, using the matrix above as a filter matrix. Save the result as a new
array ImJPG Red. Display the image.
Variables: ImJPG Red
Q2: What does the transformation above do?

7. Let us permute the colors in the image. To do this repeat the steps above with the matrix:
 
0 0 1
P ermuteM atrix =  0 1 0 
1 0 0

Save the result as a new array ImJPG Permute. If you did everything correctly, you should
see the image shown on the Fig. 6. Observe that the flowers turned purplish blue. The

26
 matrix above is an example of a hue rotation filter. You can produce other hue rotation
effects with this more general transformation:
   
0.213 0.715 0.072 0.787 −0.715 −0.072
HueRotateM atrix =  0.213 0.715 0.072  + cos(θ)  −0.213 0.285 −0.072  +
0.213 0.715 0.072 −0.213 −0.715 0.928
 
0.213 −0.715 0.928
sin(θ)  0.143 0.140 −0.283  .
−0.787 0.715 0.072
Here θ is the angle of rotation. You can try experimenting with various values of the angle
θ to get different color effects.
Variables: ImJPG Permute

Figure 6: Colors permuted

8. Now let us delete one of the colors in the image, say, green. First, produce the filter matrix
which deletes Green in the image and keeps the values of Red and Blue intact. Use it to
filter the image. Save the result as ImJPG DeleteGreen.
Variables: ImJPG DeleteGreen
9. It is possible to invert the colors of the image by using the following code:
ImJPG_Invert=255-ImJPG;
figure;
imshow(ImJPG_Invert);

Observe that here again Matlab automatically substitutes a matrix of appropriate dimen-
sion (in this case m × n × l) with all the elements equal to 255 instead of the constant 255.
Variables: ImJPG invert
10. It is also possible to enhance/mute individual colors in the image. For instance, consider
the color transformation with the matrix:
 
1.2 0 0
SaturateM atrix =  0 0.75 0 
0 0 2
Save the result as ImJPG Saturate.
Variables: ImJPG saturate
Q3: Explain what the transformation above does to the image.

27
11. Consider the color adjusting filter generated by the matrix:
 
0.7 0.15 0.15
U serM atrix =  0.15 0.7 0.15 
0.15 0.15 0.7

Apply this filter to the array ImJPG and save the result as ImJPG User.
Variables: ImJPG User
Q4: Explain what the transformation above does to the image.

12. Consider the matrix in the previous step. Is the transformation defined by this matrix
invertible? Which means, can you get the original image colors back once you applied the
filter? Or is some of the information lost? Use the inv command to create the matrix
for the inverse filter. Apply this matrix to the arrays ImJPG User and ImJPG. Display the
resulting images.
Q5: What does the inverse transformation do with the colors?

13. Repeat the steps above to see if the sepia transformation is invertible.
Q6: What do you see? Why do you think this happened?

14. Finally, try the non-linear color transformation such as Gamma-conversion:

ImJPG_Gamma1=uint8(double(ImJPG).^0.9)+30;
ImJPG_Gamma2=uint8(double(ImJPG).^1.1)-50;

Display the resulting images given by the matrices ImJPG Gamma1, ImJPG Gamma2.
Variables: ImJPG Gamma1, ImJPG Gamma2

28
Project 4: Solving linear systems in Matlab
Goals: To explore the built-in functions in Matlab used to solve systems of linear equations
Ax = b. To compare different options in terms of accuracy and efficiency.

To get started: Create a new script file and save it as lab04.m.

Matlab commands used: \, inv, rref, pinv, null, tic... toc

What you have to submit: The file lab04.m, which you will create during the lab session.

INTRODUCTION

All the methods of solution of systems of linear equations can be divided into two large
groups: direct and iterative methods. The direct methods aim to obtain an “exact” (up to
numerical round off error) solution of the system in a finite number of steps. The iterative
methods aim to create a convergent sequence of approximations to the exact solution of a linear
system. Iterative methods are usually used when the matrix of a system has a known specific
structure. Matlab has many built-in functions designed to solve systems of linear equations by
both direct and iterative methods. In this project we will discuss only direct methods such as
Gaussian elimination.
Gaussian elimination in Matlab is based on LU - or P LU -decomposition of the matrix of the
system A. We can reduce the matrix A to an upper triangular form using forward elimination
and then solve the resulting triangular system by backward substitution. In the process of
doing this, we factor the matrix A into a product of two matrices: a lower triangular matrix
L and an upper triangular matrix U. Observe that this process cannot be always carried
out without interchanges of rows, and, hence, we need an additional permutation matrix P in
our matrix decomposition. Observe, that the matrix P is necessary even in the cases when
LU -decomposition can be carried out due to the matters of numerical stability (this is called
pivoting). Gaussian elimination is the preferred method of solving the system numerically if you
do not have any additional information about the structure of the matrix A.
If the matrix A is square, we can define the inverse matrix of A (denoted as A−1 ). The
matrix A−1 is also a square matrix of the same size as A. The inverse matrix exists as long as
A is not singular, and satisfies the following equality:

A · A−1 = A−1 · A = I,

where I is the identity matrix of the same size as A and A−1 . If the matrix A is invertible,
then we can multiply both sides of the system Ax = b by the inverse matrix A−1 on the left,
and obtain the following formula for the solution of the system: x = A−1 b. However, as it
often happens in numerical linear algebra, this theoretical formula is inconvenient for numerical
computation of the solution. The reasons for this are two-fold. Firstly, the computation of inverse
matrix is significantly more costly than simply solving the system. Secondly, the accuracy of
the resulting solution will be worse than if LU -decomposition is used.
Finally, yet another possibility to solve a linear system is to use the reduced row echelon
form of the augmented matrix of the system. Theoretically, this is equivalent to using Gauss-
Jordan elimination. This method of solution will be explored in this project as well. Analysis
of the corresponding algorithm shows that the resulting solution is likely to have a larger error
compared to Gaussian elimination and is likely to take more time to compute.

29
 TASKS

1. Create a new cell and type in the following commands to generate the matrix A and the
column vector b:
%% Matlab backslash solver
clear;
A=magic(5);
b=[10; 26; 42; 59; 38]

Variables: A, b

2. If nothing is known about the structure of the matrix of the system, the best approach to
solution is to use the Matlab backslash “\” solver:
x=A\b

In most cases, this method will produce the numerical solution of the system Ax=b with
the highest possible accuracy and in the shortest computational time.
Variables: x
3. Whenever we are talking about the accuracy of the solution, ideally we want to find the
difference between the approximate numerical solution xnum and the exact solution xex
of the system. However, this is not feasible in most situations since the exact solution
xex is generally unknown (otherwise, there is no point in using numerical methods at all).
The next best thing is to find the residual r=A*x-b. If the residual is small then we can
assume, with certain limitations, that our approximate solution is close enough to the
exact solution (this is not always true, however). Let us check how accurate the solution
is which we found in the previous step. Type the following line into your Matlab script:
r=A*x-b

The resulting vector r should be very close to zero.

Variables: r
4. The Matlab backslash “\” operator is based on LU - or P LU decomposition of the matrix
A. The main idea here is that any square non-singular matrix A (meaning that the inverse
matrix A−1 exists) can be written as A = PLU (actually, in Matlab it is PT LU), where P
is a permutation matrix (or matrix obtained by permuting the rows of the identity matrix
of the same size as A); L is a lower-triangular matrix (all the elements above the main
diagonal are equal to zero), and U is an upper triangular matrix (all the elements below
the main diagonal are equal zero). Matlab has a special lu function to generate a PT LU
decomposition of the matrix A. Type the following lines into your Matlab script:
%% LU-factorization
[L,U,P]=lu(A);

This will create a new code cell and compute the matrices P, L, and U. Now let us use these
matrices to solve the linear system Ax = b:
x1=U\(L\(P*b))
r1=A*x1-b
err1=x-x1

30
 Observe that the difference between the solutions err1=x-x1 is exactly the zero vector3 .
This is due to the fact that we found the solution using exactly the same steps as Matlab.
Observe also that at no point did we use the Matlab inverse matrix function inv. In
general, there is almost never a good reason to compute an inverse matrix. Whenever you
see an inverse matrix in a numerical linear algebra textbook it is usually shorthand for
“solve a system of linear equations here”.
Variables: L, U, P, x1, r1, err1

5. Matlab also has a separate slash operator / to solve linear systems of the type yA = b.
Try running the following command
y=b’/A

Variables: y

6. Next, let us use the inverse matrix to solve the same system of linear equations:
%% Inverse matrix
x2=inv(A)*b
r2=A*x2-b
err2=x-x2

Here inv(A) produces the inverse A−1 of the matrix A. This code will also produce the
solution of the system Ax = b, compute the residual, and find the difference between
the current solution and the solution obtained by the backslash operator. Observe that
the vector of difference err2 is not exactly zero unlike in the case when we used LU -
decomposition. Observe also that the residual r2 is likely to be larger than the one
obtained with the help of the backslash operator.
Variables: x2, err2, r2

7. Finally, another way of solving the system Ax = b is based on the reduced row echelon
form. The reduced row echelon form in Matlab can be produced by using the command
rref. Type in the following code:
%% Reduced row echelon form
C=rref([A b]);
x3=C(:,6)
err3=x-x3
r3=A*x3-b

Observe that the residual r3 is much larger than the residual r. Generally speaking, rref
is also a much less efficient method of solving linear systems compared to the backslash
operator and its use for this purpose is not advised.
Variables: x3, err3, r3

8. To compare the computational efficiency of all three methods, run the code cell:
clear;
%% Computational time
Num=500;
A=rand(Num,Num)+Num*eye(Num);
3
This has been observed in both Matlab 2015a and 2019a. It has come to author’s attention that this might
not be true in other versions of Matlab.

31
 b=rand(Num,1);
tic;
x1=A\b;
toc;
tic;
x2=inv(A)*b;
toc;
tic;
C=rref([A b]);
x3=C(:,101);
toc;

This code compares all three methods in terms of the computational efficiency in the
example of a large matrix A with the dimensions Num x Num. The parameter Num in this
code is set to be 500. The matrix A is chosen in such a way that it is diagonally row-
dominant. Then by the theorem of linear algebra it is guaranteed that the matrix A is non-
singular. The Matlab function tic ... toc returns the computational time necessary
to perform the Matlab operations between tic and toc. You can try to experiment with
the parameter Num to see how the computational time changes with the size of the matrix
A. Observe that solving the system with the rref function may take a very noticeable
amount of time (you probably don’t want to set Num to be more than 1000).
Variables: Num, A, b, C, x1, x2, x3

9. Finally, let us see how to use Matlab to solve rectangular systems of linear equations. Let
A be an m × n matrix of the system and let b be an m × 1 column vector. Let us start
with the situation when m > n. In this case, the system is overdetermined. However, it is
still possible to use the Matlab backslash operator. Consider the following code:
%% Overdetermined system
clear;
A=[1 2 3; 4 5 6; 7 8 10; 9 11 12]
b=[1; 2; 3; 4]
x=A\b
r=A*x-b

The system Ax = b is inconsistent. The resulting “solution” x is not the solution of the
original linear system Ax = b itself (observe that the residual r is not close to zero), but
rather the solution to the so-called least squares problem. Namely, x minimizes the length
of the residual vector r = Ax − b. It is known that the solution to the least squares
problem can also be obtained by solving the system of the so-called normal equations
AT Ax = AT b. To verify that the vector x is indeed the solution to this system, type in
the following code:
y=(A’*A)\(A’*b)
err=x-y

and notice that x-y is indeed close to zero (it is rare to obtain an exact zero in numerical
computations due to roundoff errors).
Variables: A, b, x, r, y, err

10. Let us look at the last possible case when m < n. This is a case of an underdetermined
system. If this system is consistent then it has an infinite number of solutions. Consider
the following code:

32
 %% Underdetermined system
clear;
A=[1 2 3; 4 5 6; 7 8 9; 10 11 12]’
b=[1; 3; 5]
x=A\b
r1=A*x-b

The backslash operator in this case produces only one particular solution to the system
Ax = b. Another particular solution can be obtained by the following operations:
y=pinv(A)*b
r2=A*y-b

where the function pinv produces the so-called Moore-Penrose pseudoinverse (the precise
definition of this matrix is outside of the scope of this project). It is known that the vector
y has the smallest possible length among all the solutions of the system Ax = b. Finally,
to obtain the complete set of the solutions of the system above, consider the function
null. This function produces the complete set of the linearly independent solutions to
the homogeneous system Ax = 0. The complete solution to the inhomogeneous system
Ax = b then can be obtained by:
z=null(A)*C+x;

where the constant vector C = [C1 , C2 ]T can be chosen arbitrarily (pick any numbers for
C1 , C2 that you like!).
Variables: A, b, x, r1, y, r2, C, z

33
Project 5: Systems of linear equations and college football team
ranking (with an example of the Big 12)
Goals: To use systems of linear equations to develop a ranking for Big 12 college football teams
based on 2016-2017 season data using Colley’s and Massey’s methods.

To get started:

• Open a new Matlab script and save it as lab05.m.

• Download the files Big12th.xls, Scores.mat, and Differentials.mat. Observe that

the file Big12th.xls is provided for reference only and won’t be used explicitly in the lab.
You can also use the scores from your favorite tournament. The scores here are based on
a 10 team round-robin tournament.

Matlab commands used: load, eye, diag, sum, abs, fprintf, for ... end,
sort

What you have to submit: The file lab05.m, which you will create during the lab session.

INTRODUCTION

The present project deals with the ranking of competing sports teams. At first, this does not
appear to be a difficult problem. For instance, the easiest way to rank sports teams’ performance
is to compute the percentage of games which each team has won. This method has several
drawbacks which are especially noticeable for the college football championship. In particular,
for the percentage ranking to be objective, it is necessary that every team plays with a majority
of teams participating in the competition. However, the number of teams participating in the
Bowl Championship Series (BCS) is very large and each team only plays a small number of
games. This is what makes the ranking of college football significantly different from, say, the
NFL, where a relatively small number of teams play a representative number of games against
each other.
It is necessary to note that there is no one true answer to a ranking problem. The rank of a
team will depend on the factors which we will take into account and on the weight we will give
to those factors. There are several questions to ask here. For instance, do point differentials
matter? Is a 60-20 win any better than a 34-33 win? Taking into account point differentials may
have both pros and cons. A 34-33 win versus a strong opponent is arguably better than 60-20
versus an unranked team. At the same time, if all the teams have about the same strength, than
60-20 is clearly better. At the end of the day, whether to take into account point differentials
or not is a matter of taste and preference.
In this lab we will look at the two methods of ranking currently used by BCS with an
example of the Big 12 Conference data from 2016-2017 season. As you may know, 10 teams play
in the conference: Baylor University, Iowa State University, University of Kansas, Kansas State
University, University of Oklahoma, Oklahoma State University, Texas Christian University,
University of Texas Austin, Texas Tech University, and West Virginia University. We will look
at the games only within the Big 12 conference and ignore all other games outside of it.

34
Colley’s bias free ranking
The first method we will look at has been developed by astrophysicist Wesley Colley and is
named after him: Colley’s bias free ranking. This method does not take into account point
differentials in the games (which again can be both a good and a bad thing).
There are two main ideas behind the method. First of all, we substitute the winning per-
centages by the Laplace’s rule of succession:
1 + wi
ri = ,
2 + ti
where wi is the number of winning games for the ith team, ti is the number of total games,
and ri is the rank of the team. Laplace’s rule of succession is used for events which have only
two outcomes such as either winning or losing. The rule comes from the idea in probability
that unless an event always happens or never happens, we should avoid assigning this event
probabilities of 1 (always true) and 0 (never true). Indeed, after playing one game, a team could
either win the game or lose it. If the team won the game, than its percentage ranking is 100%;
if it lost the game, its ranking is 0%. Meaning that after one game some teams are infinitely
better than others which is not very realistic. At the same time, the Laplace’s rule of succession
will give us approximately 2/3 and 1/3, meaning that the team which won one game is about
two times better than the one which lost one game - a good starting point for further estimates.
The second idea is that the rank of opposing teams should matter just as much as the total
number of wins and losses. To incorporate the opposing teams’ rankings into our system, we
can write:
wi − li wi + li wi − li ti
wi = + = + ,
2 2 2 2
where li is the number of losses for the ith team.
Now observe that with the use of Laplace’s succession rule the rankings of the teams will
hover around 1/2, so we can approximate:

i t
ti 1 X
= ti ≈ rji .
2 2
j=1

Basically, we substitute each 1/2 with the ranking of the rji of the jth team which ith team
played against. Now, let us substitute the last formula back into the Laplace’s rule to obtain
the following system of linear equations (check it!):
ti
X wi − li
(2 + ti )ri − rji = 1 + .
2
j=1

This system has N equations with N unknowns ri , where N is the number of teams playing
in the championship (in our case N = 10). Observe that the matrix of this system has special
properties - it is diagonally row dominant, meaning that the system will always have a unique
solution.

Massey’s method
Massey’s method was developed by Ken Massey in his undergraduate research project and is
now used by the BCS (Oh, the places you can go!... if you know math). Unlike Colley’s method,

35
Massey’s takes into account point differentials as well as number of wins/losses. Again, denote
as ri the rankings of the competing teams. The data we obtain from the games can be written
in the form of linear equations:
ri − rj = dij ,
meaning that ith team won (lost to) jth team by the dij point differential.
The data from the games produces a large system of linear equations of the form:

PR = B,

where P is the matrix of the system such that each row has only two non-zero elements, 1 and
−1, R is the column-vector of the rankings ri , and the vector of the right-hand-side B contains
the corresponding point differentials.
Observe that, in general, we are likely to have a lot more equations than unknowns (more
games than teams), so, in general, this system of equations is likely to be inconsistent and not
have any solutions. To see this, assume that the first team beat the second team by 20 points
and the third team beat the second team by 5 points. This data implies that the first team
should beat the third team by 15 points. Of course, the point differential of the actual game (if
it happens!) between the first team and the third team might be different.
So, instead, we will give up on trying to solve this system exactly, and we will look for the
vector R which satisfies the system as close as possible. Namely, we will employ the least squares
method, meaning, that we want to find the vector R which minimizes the vector norm:

||PR − B||2

where || · ||2 is a usual Euclidean norm: if x = (x1 , x2 , ..., xn ) is a vector with n components,
then q
||x||2 = x21 + x22 + ... + x2n .
It is well-known from linear algebra that this problem is equivalent to the so-called system
of normal equations written as:
PT PR = PT B,
where PT is a transpose of the matrix P. Now, PT P is an N × N matrix, where N is again the
number of the teams, so this results in a square system of linear equations.
The last obstacle which we will need to overcome is that the matrix PT P has a rank of at
most N − 1 (why?), and hence, in general, the last system does not have a unique solution. To
obtain a unique solution we will change each element in the last row of the matrix PT P to 1
and we will change the last element in the vector PT B to zero. Practically this means that all
the rankings ri obtained in this way will need to sum up to zero.
Now that you learned the theory behind the ranking, let us create code in Matlab which will
produce the rankings of teams in the Big 12. To do so, first of all save the data files Scores.mat
and Differentials.mat in the directory you will be working in. Each of these files contains
the results of the games from the Big 12 championship from the season 2016-2017 in the form
of the table (matrix). The Scores.mat contains a 10 × 10 matrix where each row corresponds
to the scores of the Big 12 teams in the following order:
1. Baylor

2. Iowa State

3. University of Kansas

36
 4. Kansas State

5. University of Oklahoma

6. Oklahoma State

7. Texas Christian

8. University of Texas Austin

9. Texas Tech

10. West Virginia

The element (i, j) of the matrix Scores is 1 if ith team won over jth team. Correspondingly, if
ith team lost to jth team, then the (i, j) element is −1. The diagonal elements are taken to be
0.
The file Differentials.mat contains the matrix Differentials which is similar to Scores,
only instead of ±1 it has the point differentials of the corresponding games.

TASKS

1. Start by loading both matrices into your Matlab script using the load command:
load(’Scores.mat’);
load(’Differentials.mat’);

Variables: Scores, Differentials

2. First, consider the Colley method. Create the matrix and the right-hand side of Colley’s
system of linear equations:
%% Colley’s method
games=abs(Scores);
total=sum(games,2);
ColleyMatrix=2.*eye(10)+diag(total)-games;
RightSide=(1+0.5.*sum(Scores,2));

Variables: ColleyMatrix, RightSide

Q1: What do the commands eye and diag do?

3. Solve the linear system using the backslash operator:

RanksColley=ColleyMatrix\RightSide;

Variables: RanksColley

4. Display the results on the screen:

Teams={’Baylor’, ’Iowa State’, ’University of Kansas’, ’Kansas State’,...
’University of Oklahoma’, ’Oklahoma State’, ’Texas Christian’,...
’University of Texas Austin’, ’Texas Tech’, ’West Virginia’};
[RanksDescend,Order]=sort(RanksColley,’descend’);
fprintf(’\n’);
for j=1:10
fprintf(’%8.3f %-15s\n’, RanksColley(Order(j)), Teams{Order(j)})
end;

37
 Variables: Teams, Order, RanksDescend
Q2: Observe that the array Teams is in curly parentheses. What kind of array is that?
Q3: Explain what kind of formatting does the line ’%8.3f % − 15s \ n’ in the command
fprintf produce?
Q4: What does sort function do?

5. Now let us use Massey’s method to rank the teams. Create the matrix P and the right-
hand-side B by using the following code:
%% Massey’s method
l=0;
for j=1:9
for k=j+1:10
if Differentials(j,k)~=0
l=l+1;
P(l,j)=1;
P(l,k)=-1;
B(l)=Differentials(j,k);
end;
end;
end;

This code scans the part of the matrix Differentials which is located above the main
diagonal. For each nonzero (i, j) (meaning that the teams i and j played a game) element
of this matrix it creates a row of the matrix P which contains only two non-zero elements:
1 in the ith column of this row and −1 in the jth column. The vector B of the right-hand-
side is created simultaneously and has the corresponding point differential in the game
between ith and jth team.
Variables: P, B
6. Next, create the normal system of linear equations:
A=P’*P;
D=P’*B’;

Variables: A, D
7. Substitute the last row of the matrix as described above:
A(10,:)=[1 1 1 1 1 1 1 1 1 1];
D(10)=0;

8. Finally, solve the system:

RanksMassey=A\D;

Variables: RanksMassey

9. Display the results on the screen as before:

[RanksDescend,Order]=sort(RanksMassey,’descend’);
fprintf(’\n’);
for j=1:10
fprintf(’%8.3f %-15s\n’, RanksMassey(Order(j)), Teams{Order(j)})
end;

38
 Variables: RanksDescend, Order

10. Compare the results of both Colley’s and Massey’s methods. Compare both the methods
with the official results. Observe that while these two methods are used by BCS, they do
not take into account many things, such as the timing of the games, strength of schedule,
games played against opponents which are not in the Big 12, and so on. These factors
may affect the rankings and make it deviate from the official. For the data considered in
this project the comparison with the official results should be very good.

11. Create a new code cell. Switch the result of the game between the top two teams in
Colley’s ranking and run the Colley’s algorithm again. Produce the rankings of the teams
in this case.
Q5: Did the ranking of the top teams change?

12. Create a new code cell. Now, switch the result of the game between two top teams in
Massey’s ranking (those might be different teams from Colley’s algorithm!) and run the
Massey’s algorithm again. Produce the rankings of the teams in this case.
Q6: Did the ranking of the top two teams change?

39
Project 6: Convolution, inner product, and image processing re-
visited
Goal: In this project we will look at the convolution of two matrices and its application to
image processing.

To get started:
• Create a new Matlab script file and save it as lab06.m

• Download the file einstein.jpg and put it in your working directory.

Matlab commands used: load, rand, double, uint8, conv2, filter2

What you have to submit: The file lab06.m which you will create during the lab session.

INTRODUCTION

In this section we will look at the applications of matrix multiplication to the filtering of
images. The most common operations in filtering are bluring/smoothing, sharpening, and edge
detection. All of these are examples of so-called convolution filters. These filters work by
changing the color of the pixels in the image using the colors of nearby pixels.
Let us look at how filters of this type work. Consider an image described by the matrix
f = (fij ) where (i, j) are integer coordinates of the pixel in an image, and the value fij returns
the color of the image. Consider h = (hkl ) to be a convolution matrix (filter matrix). These
matrices might have (and usually do have) different dimensions. The filter matrix is usually
relatively small. Then a new matrix which is a convolution of f and h (written as g = f ? h)
can be found by the following formula:
XX
gij = fkl hi−k+1,j−l+1 ,
k l

where the summation runs over all the indices which make sense. In particular, if an element
with the certain index does not exist, we simply assume the corresponding term to be zero. You
can think of convolution being obtained by the filter matrix h sliding over the original image f .
Notice that the formula for convolution resembles the definition of an inner product if we
treat both matrices as vectors (say, with columnwise orientation) and flip the second matrix
both horizontally and vertically.
Matlab has two functions to compute convolutions of matrices: conv2 and filter2. These
two functions are closely related. In particular, the function filter2 is equivalent to the function
conv2 with the matrix h rotated 180 degrees.

TASKS

1. To begin, let us load the file einstein.jpg and save the resulting matrix as ImJPG. Use
the size function to find the dimensions m, n of the matrix ImJPG.
Variables: ImJPG, m, n
2. Introduce some noise into the image by adding random fluctuations of color to each point:
ImJPG_Noisy=double(ImJPG)+50*(rand(m,n)-0.5);

40
 Observe that the command rand(m,n) produces a matrix of the dimension m × n filled
with pseudorandom numbers within the interval (0, 1). The amplitude of the noise is equal
to ±25 shades of gray. Also, notice the function double which converts the variables from
the type uint8 to the type double.
3. Two of the most common operations on images done with convolution filters include
smoothing and sharpening. Let us start by using two average smoothing filters given
by the matrices:
   
0 1 0 1 1 1
1 1
Kernel Average1 = 1 1 1  , Kernel Average2 =  1 1 1 
5 9
0 1 0 1 1 1

4. Using the appropriate Matlab syntax, type in the matrices Kernel Average1 and
Kernel Average2.
Variables: Kernel Average1, Kernel Average2
Q1: Why are the fractions 1/5 and 1/9 necessary?

5. Apply the filters to the noisy image by using the commands

ImJPG_Average1=conv2(Kernel_Average1,ImJPG_Noisy);
ImJPG_Average2=conv2(Kernel_Average2,ImJPG_Noisy);

Display the resulting images in separate figure windows and observe the result. Don’t
forget to convert the results back to the integer format by using uint8 function.
Variables: ImJPG Average1, ImJPG Average2
Q1: What is the size of the matrices ImJPG Average1, ImJPG Average2?
Q2: What is the size of the original matrix ImJPG? Use the size function.
Q3: Which filter blurs more? Why?
6. An alternative blurring filter, Gaussian blur, is given by the matrix:
 
0 1 0
1
Kernel Gauss = 1 4 1 
8
0 1 0
which assigns a higher weight to the pixel color in the center. Type in the matrix
Kernel Gauss in your Matlab file using the appropriate syntax.
Variables: Kernel Gauss
7. Perform the convolution using the function conv2 and Kernel Gauss and save the resulting
array as ImJPG Gauss. Display the result in a new window.
Variables: ImJPG Gauss
8. Observe that we can “layer” filter effects. Perform another convolution with the Gaussian
kernel on the image ImJPG Gauss, save the result as ImJPG Gauss2, and display the result
in a new window.
Variables: ImJPG Gauss2
Q4: Devise a matrix for a filter which is equivalent to applying Gaussian convolution
twice.
Q5: What is the size of the matrix for this filter?

41
 9. It is also possible to blur the image over a larger area:
 
0 1 2 1 0
 1 4 8 4 1 
1 
 2

Kernel Large = 8 16 8 2 
80 
 1

4 8 4 1 
0 1 2 1 0

Apply the kernel Kernel Large to the matrix ImJPG, save the result as ImJPG Large, and
display the figure in a new figure window.
Variables: ImJPG Large
Q6: Compare with applying Gaussian blur applied twice. Which one blurs more?

10. An opposite action to blurring is the sharpening of an image with a convolution filter.
Type the following kernels into your Matlab file:
   
0 −1 0 −1 −1 −1
Kernel Sharp1 =  −1 5 −1  , Kernel Sharp2 =  −1 9 −1  .
0 −1 0 −1 −1 −1
Variables: Kernel Sharp1, Kernel Sharp2

11. Perform the convolution of the original image ImJPG with the kernels Kernel Sharp1,
Kernel Sharp2 using the function conv2 and save the resulting arrays as ImJPG Sharp1
and ImJPG Sharp2. Display the results in new figure windows.
Variables: ImJPG Sharp1, ImJPG Sharp2

12. Finally, it is possible to use convolution to detect edges in the image. Edge detection
is used for image segmentation and data extraction in areas such as image processing,
computer vision, and machine vision. Two of the most common filters used for these are
the Sobel horizontal and vertical filters:
   
−1 0 1 −1 −2 −1
Kernel Sobel1 =  −2 0 2  , Kernel Sobel2 =  0 0 0 .
−1 0 1 1 2 1

Sobel filters can be interpreted as discrete derivatives in the horizontal and vertical direc-
tions. Type in the matrices Kernel Sobel1, Kernel Sobel2.
Variables: Kernel Sobel1, Kernel Sobel2

13. Perform the convolution of the original image ImJPG with the Sobel kernels using the func-
tion conv2 and save the resulting arrays as ImJPG Sobel1 and ImJPG Sobel2. Display the
results in new figure windows.
Variables: ImJPG Sobel1, ImJPG Sobel2

14. Create a combined image with both horizontal and vertical edges by summing up the
matrices ImJPG Sobel1, ImJPG Sobel2. Display the result in a new figure window by
using the code:
figure;
imshow(uint8(ImJPG_Sobel1+ImJPG_Sobel2));

42
15. Alternatively, Laplacian edge detection can be used with the following filter:
 
0 −1 0
Kernel Laplace =  −1 4 −1  .
0 −1 0

Type in the matrixKernel Laplace. The laplace kernel is a discrete analogue of continuous
Laplacian and may be interpreted as a sum of two discrete partial derivatives of the second
order.
Variables: Kernel Laplace

16. Perform the convolution of the original image ImJPG with the Laplace kernel using the
function conv2 and save the resulting array as ImJPG Laplace. Display the results in a
new window.
Variables: ImJPG Laplace

43
Project 7: Norms, angles, and your movie choices
Goals: Use a norm and an inner product to detect similarity between users’ choices and to
create a simple recommender system.

To get started:

• Open a new Matlab script and save it as lab07.m.

• Download the file users movies.mat4 which contains the arrays movies, users movies,
users movies sort, index small, trial user and place it in your working directory.

Matlab commands used: load, size, for... end, if... end, fprintf, length,
prod, norm

What you have to submit: The file lab07.m which you will create during the lab session.

INTRODUCTION

Being able to predict users’ choices is big business. Many internet services are studying
consumer choices and preferences to be able to offer you the products you might like (and
correspondingly the products you are more likely to buy). Netflix, Pandora Internet radio,
Spotify, Facebook targeted ads, Amazon, and even online dating websites are all examples of
recommender systems. These services are sometimes willing to go to great lengths in order to
improve their algorithms. In fact, in 2006, Netflix offered 1 million dollars to the team which
would be able to improve their Cinematch algorithm predictions by just 10%.
Music Genome Project which is a trademark of Pandora Internet radio is another example of
a recommender system. If you have ever listened to Pandora, you might have been amazed how
uncannily spot-on their next song recommendation is (at least, the author of this project was
certainly impressed by it). Every song is assigned a rating from 0 to 5 in 450 different categories
(“genes”). Thus, any song is represented by a vector with 450 components. Mathematical
comparison of these vectors allows the algorithm to suggest the next song to a user.
Observe that data about movies, songs, consumer products, etc, can be often arranged
in a form of a vector. These vector representations can then be used in different algorithms
to compare items and predict similarity. In this project, we will use linear algebra to establish
similarities in tastes between different users. The main idea is that if we know the current ratings
from a certain user, then by comparing them with the ratings of other users in a database, we
can find users with similar tastes. Finally, we can look at the items highly rated by those users
and suggest these items to the current user.
In this lab, we will look at the MovieLens database [10] which contains around 1 million
ratings of 3952 movies by 6040 users. We will create a very simple recommender system by
finding similarities between users’ tastes. Our techniques will be based on using the Euclidean
distance (norm), the standard scalar product, and finding correlation coefficients (or angles
between the vectors of data).

4
The original data is available at https://grouplens.org/datasets/movielens/.

44
 TASKS

1. Load the arrays movies, users movies, users movies sort, index small, trial user
from the file users movies.mat using the load command. The matrix users movies
should be a 6040 × 3952 matrix containing integer values between 0 and 5 with 1 meaning
“strongly dislike” and 5 meaning “strongly like”. The 0 in the matrix means that the user
did not rate the movie. The array movies contains all the titles of the movies. The matrix
users movies sort contains an extract from the matrix users movies.mat with rating
for the 20 most popular movies selected. The indexes of these popular movies are recorded
in the array index small. Finally, ratings of these popular movies by yet another user
(not any of the users contained in the database) are given by the vector trial user. It is
suggested to view all the variables and their dimensions by using the “Workspace” window
of the Matlab environment:
%% Load the data
clear;
load(’users_movies.mat’,’movies’,’users_movies’,’users_movies_sort’,...
’index_small’,’trial_user’);
[m,n]=size(users_movies);

Variables: movies, users movies, users movies sort, index small, trial user, m, n

2. Print the titles of the 20 most popular movies by using the following code:
fprintf(’Rating is based on movies:\n’)
for j=1:length(index_small)
fprintf(’%s \n’,movies{index_small(j)})
end;
fprintf(’\n’)

Observe that the movie titles are called from the cell array movies (notice the curly
parentheses) by using an intermediate array index small.
3. Now let us select the users we will compare the trial user to. Here, we want to select the
people who rated all of the 20 movies under consideration. This means that there should
not be any zeros in the corresponding rows of the matrix users movies sort. This can
be accomplished by the following code:

%% Select the users to compare to

[m1,n1]=size(users_movies_sort);
ratings=[];
for j=1:m1
if prod(users_movies_sort(j,:))~=0
ratings=[ratings; users_movies_sort(j,:)];
end;
end;

Observe the use of the command prod to compute the product of the elements of the rows
of the array users movies sort. The array ratings contains all the users which have
rated all the popular movies from the array index small. In general, it may happen that
this array is empty or too small to create any meaningful comparison - we won’t consider
these cases in this project.
Variables: ratings, m1, n1
Q1: What does the command ratings=[] do?

45
4. Next, we can look at the similarity metric based on the Euclidean distance. The idea here
is that we treat the array trial user and the rows of the array ratings as vectors in 20-
dimensional real space R20 . Assume that all the vectors have the origin as the beginning
point. We can find the distance between the end points of the vector trial user and each
of the vectors ratings(j,:). In other words, we are looking for the user with the closest
ratings to the trial user. This can be accomplished by the following code:

%% Find the Euclidean distances

[m2,n2]=size(ratings);
for i=1:m2
eucl(i)=norm(ratings(i,:)-trial_user);
end;

The vector eucl contains all the Euclidean distances between trial user and the rows of
the matrix ratings.
Variables: eucl
5. Now let us select the user from the database with the smallest Euclidean distance from
the trial user. Instead of using the usual function min we will use a slightly more com-
plicated approach. Let us sort the elements of the vector eucl by using the function sort.
The advantage of this is that it allows us to find the second closest user, the third closest
user, etc. There may only be a small difference between the several closest users and we
might want to use their data as well.

[MinDist,DistIndex]=sort(eucl,’ascend’);
closest_user_Dist=DistIndex(1)

Variables: MinDist, DistIndex, closest user Dist

6. The similarity metric above is one of the simplest ones which can be used to compare two
objects. However, when it comes to user ratings it has certain disadvantages. For instance,
what if the users have similar tastes, but one of them consistently judges movies harsher
than the other one? The metric above would rate those two users as dissimilar since the
Euclidean distance between vectors of their opinions might be pretty large. To rectify this
problem, we can look at a different similarity metric known in statistics as the Pearson
correlation coefficient which can be defined as
PN
i=1 (xi −q x̄)(yi − ȳ)
r(x, y) = qP ,
N 2
PN 2
(x
i=1 i − x̄) (y
i=1 i − ȳ)

where x = (x1 , x2 , . . . , xN ), y = (y1 , y2 , . . . , yN ) are two vectors with N components, and

x̄ = N1 N 1 PN
P
i=1 xi , ȳ = N i=1 yi are the average (mean) values of the vectors x and y. If we
look at the formula for r(x, y), we can see that it accounts for the average user opinion x̄,
ȳ, and also for how harshly/enthusiastically they tend to judge their movies (defined by the
magnitude of the vectors x−x̄ and y−ȳ, where x̄ and ȳ denote the vectors of the same size
as x and y with all the components equal to x̄ and ȳ correspondingly). Geometrically, the
correlation coefficient r(x, y) corresponds to the cosine of the angle between the vectors
x − x̄ and y − ȳ (the closer this angle is to zero, the more similar are the opinions of two
people). To compute the Pearson correlation coefficient, let us centralize the columns of
the matrix ratings and the vector trial user first:

46
 ratings_cent=ratings-mean(ratings,2)*ones(1,n2);
trial_user_cent=trial_user-mean(trial_user);

Variables: ratings cent, trial user cent

7. Next, use the for... end loop to compute the Pearson correlation coefficients between
the rows of the matrix ratings and the vector trial user. Save the result as a vector
pearson.
Variables: pearson

8. Finally, observe that the value r(x, y) belongs to the interval (−1, 1). The closer the
coefficient is to 1, the more similar the tastes of the users are. Finally, let us sort the
vector pearson as before using the sort function. Save the results of this function as
[MaxPearson,PearsonIndex] and find the maximal correlation coefficient which will be
the first element in the matrix MaxPearson. Save this element as closest user Pearson.
Variables: MaxPearson, PearsonIndex, closest user Pearson
9. Compare the elements of the vectors DistIndex, PearsonIndex.
Q2: Are the variables closest user Pearson and closest user Dist the same?

10. Now let us display the recommendations on the screen. Use the following list to create
the list of movies which the trial user has liked and the lists of recommendations for
him/her based on the distance criterion and the Pearson correlation coefficient criterion:

%% Recommendations
recommend_dist=[];
for k=1:n
if (users_movies(closest_user_Dist,k)==5)
recommend_dist=[recommend_dist; k];
end;
end;
recommend_Pearson=[];
for k=1:n
if (users_movies(closest_user_Pearson,k)==5)
recommend_Pearson=[recommend_Pearson; k];
end;
end;
liked=[];
for k=1:20
if (trial_user(k)==5)
liked=[liked; index_small(k)];
end;
end;

We use the rating equal to 5 both as the criterion for liking the movie and the criterion
to recommend the movie. Of course, you can broaden this up and also include the movies
ranked as 4.
Variables: liked, recommend dist, recommend Pearson
11. Finally, display the titles of the movies from the arrays liked, recommend dist,
recommend Pearson on the screen by using the procedure similar to the one used in the
step 2.

47
12. Take a look at the list of the popular movies displayed in step 2. Chances are you might
have seen the majority of them before. Create your own vector of ratings and call it
myratings. Ratings should be integers between 1 and 5. Again, assign the rating 1 if
you really disliked the movie, and 5 if you really liked it. If you haven’t seen a particular
movie, pick its rating at random. The vector of ratings myratings should be a row-vector
with 20 elements.
Variables: myratings

13. Create a new code cell. In this cell, repeat steps 4-11 of this project and substitute the
vector trial user by the vector myratings. This should produce a personal recommen-
dation based on your own ratings.
Variables: liked, recommend dist, recommend Pearson

48
Project 8: Interpolation, extrapolation, and climate change
Goals: To explore applications of linear algebra for the interpolation of data and prediction of
future tendencies.

To get started:

• Create a new Matlab script and save it as lab08.m.

• Download the file temperature.mat which contains average temperatures for the past 136
years 5 . Temperature is given in Celsius.

Matlab commands used: plot, axis, vander, interp1, orth

What you have to submit: The file lab08.m, which you will create during the lab session.

INTRODUCTION

In many practical applications it is necessary to obtain an intermediate value of a parameter

from a few known data points. If this intermediate value lies in between two known measure-
ments, we will call this process interpolation. If the value is outside of the sampled interval,
then we will call this process extrapolation. In this project we will look at several procedures
for interpolation and extrapolation and apply them to weather and climate change prediction.
The most common reason for using interpolation is not having enough of data points. These
data points can be obtained from experimental observations which can be expensive or difficult
to perform, or from past observations where the data is simply unavailable. Interpolation is
also used when the function which produces the data is known but has a complicated struc-
ture. In this case it can be computationally prohibitive to use this function, especially if, in
the course of computation, this function needs to be called multiple times. For instance, the
function can be given by a slowly convergent series which requires computing many terms to
obtain adequate accuracy. Approximating this complicated function by a simpler function may
be helpful. Interpolation of a function can be sufficiently accurate provided that the function
is smooth and there are enough data points available. Some of the usual types of interpolation
include polynomial interpolation when the function is approximated by a polynomial, trigono-
metric approximation with a truncated Fourier series, or various piece-wise approximations.
More intricate interpolation methods can be used if additional information is available about
the approximated function.
Extrapolation, on the other hand, is a significantly more difficult procedure. The general
rule is not to extrapolate too far beyond the range of the observed parameters. There are
multiple reasons for this. For instance, some additional parameters may affect the behavior
of a function outside of the observed range of values. Imagine trying to extrapolate physical
laws applicable for regular macro-solids (such as things we use in our daily life) to nano-scale
or to astro-scales. The obtained results will be meaningless since different laws apply at those
scales. Greater uncertainty and errors while extrapolating also appear because small errors in
data measurements can potentially lead to very large errors while extrapolating the data. These
errors are somewhat bounded while interpolating between two known data points.
5
The temperature data has been taken from NASA’s website, https://climate.nasa.gov/vital-signs/
global-temperature/

49
 TASKS

1. To begin, let us look at interpolation based on the polynomial approximation. Assume

that we have n + 1 data points (measurements) (xi , yi ), i = 0, . . . , n. To obtain the values
of the function in between these data points we can simply trace a polynomial of the nth
degree through the data points and use this polynomial to estimate the values of a function
in between measurements. Thus, the goal is to find a polynomial

p(x) = an xn + an−1 xn−1 + . . . a1 x + a0

with unknown coefficients ai , i = 0, . . . , n, which passes through the points (xi , yi ). Then
the coefficients ai must satisfy the following system of linear algebraic equations

an xni + an−1 xin−1 + . . . a1 xi + a0 = 0, i = 0, . . . , n.

We can write this system in the form

Vz = y,

where V is the so-called Vandermonde matrix:

... xn−1 xn0
 
1 x0 0
 1 x1 ... xn−1
1 xn1 
V=  ...
,
... ... ... ... 
1 xn ... xn−1
n xnn

z = [a0 , a1 , . . . , an ]T is the unknown vector of polynomial coefficients and y = [y0 , y1 , . . . , yn ]T

is the known vector of the function values at sample points.
We will start by approximating the data for the average temperatures in Kansas6 :

Month 1 2 3 4 5 6 7 8 9 10 11 12
Temperature (High) 37 44 55 66 75 84 89 88 80 69 53 41
Temperature (Low) 17 23 33 43 53 63 68 65 56 44 32 22

In our computations, we will work with high temperatures only. To begin, let us generate
a plot of the temperatures in Matlab.
clear;
WeatherHigh=[37 44 55 66 75 84 89 88 80 69 53 41]’;
figure;
plot(1:12,WeatherHigh,’r-x’);
axis([1,12 30 95]);

Observe that Matlab will automatically join the available data points by straight lines.
Variables: WeatherHigh

2. Assume for now, that we are only given the temperatures for four months: January, May,
August, and December. Select the corresponding data from the variable WeatherHigh
above and generate the Vandermonde matrix based on this data. Now solve the resulting
system using Matlab’s backslash linear system solver:
6
Source for the temperature data: Wikipedia https://en.wikipedia.org/wiki/Kansas#Climate, temperature
is given in Fahrenheit.

50
 x=[1,5,8,12]; %Numbers for January, May, August, and December
V=vander(x);
CoefHigh=V\WeatherHigh(x);

The resulting vector CoefHigh contains coefficients of the cubic polynomial approximating
the average high annual temperature in Kansas.
Variables: V, CoefHigh

3. Matlab’s polyval function can be used to efficiently evaluate a polynomial at the given set
of points. It takes two vectors as arguments. The first vector represents the coefficients
of the polynomial and the second vector represents the points we want to evaluate the
polynomial at. Remember that you can always type help polyval or doc polyval in the
Matlab command window to see more options. Let us plot the graph of the polynomial
obtained at the previous step by using the following code:
xc=1:0.1:12;
ycHigh=polyval(CoefHigh,xc);
plot(xc,ycHigh,’b-’,1:12,WeatherHigh,’r-x’);
axis([1,12 30 95]);

This graph shows the difference between the approximated values and the exact values.
Observe that the command axis([1,12 30 95]) sets the parameters for x and y axes.
Namely, the x axis will range from x = 1 to x = 12 and the y axis will range from y = 30
to y = 95.
Variables: xc, ycHigh

4. Repeat the steps above to produce a polynomial approximation based on the temperatures
from the following six months: January, March, May, August, October, and December.
You will need to use a polynomial of the fifth degree. Plot the resulting polynomial next
to the exact values as in the step above. You will see that this approximation produces
much closer results to the exact values.
5. Finally, let us use the temperatures from all twelve months. You will need to use a
polynomial of the eleventh degree this time. Modify the code above and run it in Matlab.
If you did everything correctly, you will obtain a picture which looks similar to the one
in Fig. 7. You may notice that this approximation is not necessarily better than the

Figure 7: Approximation with the 11th degree polynomial

51
 approximation with the 5th degree polynomial due to large fluctuations near the end
points. This is typical for approximations with polynomials of a high degree on evenly
spaced grids which tend to oscillate widely near the end points. This is one of the reasons
why polynomials of a high degree are normally not used for interpolation on evenly spaced
grids 7 . Additionally, if you look at the Matlab output in the command window, you will
probably see something like this
Warning: Matrix is close to singular or badly scaled. Results may be inaccurate.
RCOND = 8.296438e-17.

This is due to the fact that Vandermonde matrix V is an ill-conditioned matrix, meaning
that the system of linear equations with this matrix is difficult to solve with high accuracy.
The matrix V becomes more ill-conditioned as the dimensions of the matrix increase. This
is another reason why high degree polynomial approximations on evenly spaced grids are
rarely used in interpolation.

6. If polynomials of a high degree on evenly spaced grids are not good for interpolation,
what can be done instead? The most common method of interpolation is through the
approximation of functions with piece-wise (typically polynomial) functions. This means
that the function between data points is approximated by a small degree polynomial
(polynomials of no higher than third degree are usually used). These polynomials are
different on each of the intervals between the data points. Individual approximations
are then glued together to achieve a desired degree of smoothness at the data points.
Under smoothness here we understand the continuity of a function and its derivatives up
to the some order. Smoothness of the resulting approximation depends on the degree of
polynomials used and how the polynomial pieces are glued together. If a higher degree of
smoothness is desired, it is necessary to use polynomials of a higher degree. For instance,
if we use cubic polynomials, then it is possible to achieve continuity of the function itself
and its first two derivatives at the most.
To investigate this method of interpolation, we will look at the built in Matlab function
interp1. This function allows one to interpolate a one-dimensional function. It has several
attributes which allow for different types of interpolation: “linear” which simply joins
the data points with straight lines, “cubic” or “pchip” which approximates the function
on each individual segment with the third degree polynomial, and “spline” which also
uses third degree polynomials, but with different junction conditions. So what is the
difference between “pchip” and “spline”? Simply put, the “pchip” method provides a
shape-preserving piece-wise cubic polynomial function which is smooth together with its
first derivative. The “spline” interpolation creates an “extra-smooth” piece-wise cubic
polynomial function which is smooth together with its first and second derivatives. In
general, “pchip” approximations have smaller oscillations near the ends and are easier to
compute.
Compare these options by using the following code:
ycHigh1=interp1(x,WeatherHigh,xc,’linear’);
ycHigh2=interp1(x,WeatherHigh,xc,’pchip’);
ycHigh3=interp1(x,WeatherHigh,xc,’spline’);
7
However, approximations with the polynomials of high degree on non-evenly spaced grids may be extremely
useful in computations and achieve very high precision. Chebyshev nodes and Chebyshev polynomials represent
a good example of this phenomena.

52
 Here the variables x and WeatherHigh give the vectors of initial data. The vector xc is
a vector of points where we want to approximate the function and the vectors ycHigh1,
ycHigh2, and ycHigh3 are the values at the points xc resulting from the interpolation.
Plot the vectors ycHigh1, ycHigh2, and ycHigh3 versus xc on the same plot to observe
the smoothness of the resulting data. You can see that ycHigh2 and ycHigh3 achieve
better approximations than the polynomial approximation. The function interp1 also
has a possibility to extrapolate the results which we will explore shortly.
Variables: ycHigh1, ycHigh2, ycHigh3

7. Now let us look at the extrapolation of the data. Extrapolation means finding an ap-
proximation to the data outside of the initial measurement interval. For our data, we will
take the average global temperatures from the last 136 years starting from the year 18808 .
To begin, load the matrix temperature from the file “temperature.mat” using the load
command.
Variables: temperature

8. The matrix temperature consists of two columns. The first column represents the years
and the second the average temperatures at those years. Separate the data into two
variables:
years=temperature(:,1);
temp=temperature(:,2);

Plot the graph of temperatures using the plot function. This will produce a graph of
fluctuations of average temperatures since 1880. If you did everything correctly, you should
obtain a graph similar to the one on the Fig. 8.
Variables: years, temp

Figure 8: Average yearly temperatures.

9. Let us attempt to extrapolate the data in the future. Assume that we want to know
the temperature about 10 years into the future (again, it is advised not to attempt to
extrapolate too far beyond the initial interval). We can do this using the following code:
futureyears=2016:2025;
8
The source of data: NASA, https://climate.nasa.gov/vital-signs/global-temperature/

53
 futuretemp1=interp1(years,temp,futureyears,’linear’,’extrap’);
futuretemp2=interp1(years,temp,futureyears,’pchip’,’extrap’);
futuretemp3=interp1(years,temp,futureyears,’spline’,’extrap’);

figure;
plot(years,temp,’b-o’,futureyears,futuretemp1,’g-o’,...
futureyears,futuretemp2,’r-x’,futureyears,futuretemp3,’k-d’);

This produces three graphs based on the different interpolation/extrapolation strategies

which were discussed above. If you did everything correctly, you should obtain three
radically different predictions for the next 10 years!
Variables: futureyears, futuretemp1, futuretemp2, futuretemp3

10. So what happened in the previous example? The local fluctuations in temperature signifi-
cantly affected the global predictions for the future. However, we are not interested in the
local fluctuations but rather in global tendencies. So let us try to separate those two from
each other and extrapolate based on global tendencies only. We can think of temperatures
in the following terms:
ti = t(yeari ) + noise,
where t(year) would be the global tendency of temperature and noise represents relatively
small year-to-year fluctuations.
We will use orthogonal projections to separate the global behavior from the “noise”. As
a first approximation, let us consider how we can assess the overall average temperature.
That is, we approximate the temperature t(years) as a constant:
n
1X
t(years) ≈ ti .
n
i=1

Enter the following command in the command window

>> sum(temp)/n

This allows us to see that the average temperature for the past 136 years was 0.0244C.

11. Here is another way of thinking of the same operation using orthogonal projections. If the
data was truly flat, then there would be a real number c so that
   
t1 1
 t2   1 
t =  .  = c  .  = c1.
   
 ..   .. 
tn 1

That is, the average temperature vector t would be a scalar times the vector 1 of all ones.
Since the data is not flat, we can’t find such a c. However, we can do the next best thing.
We can find c to minimize the norm kt − c1k. The best choice for c is determined by the
orthogonal projection on a vector 1:

1T t 1T t
c= and for this choice of c, c1 = 1 = proj1 (t).
k1k2 k1k2

54
 Figure 9: Approximation of data by a constant function.

Moreover, a little rearranging shows that

 
1 T 1
proj1 (t) = 11 t = Pt where P= 11T is the projection matrix.
k1k2 k1k2

Define the variable b1 to be the length(temp)-dimensional 1 vector. In the space after

that, define the variable P1 to hold the projection
√ matrix. You will either want to use
Matlab’s norm function, or remember that kuk = uT u. Also, define the variable temp1
to be the value P1*temp.
Variables: P1, temp1

12. Plot the resulting vector using the code:

plot(years,temp,’bo’,years,temp1,’g.’);
hold on;

You should see a similar plot as before, but now with a set of green dots representing the
projection (Fig. 9).
13. Now, examine the value of temp1 in the Matlab command window.
Q1: How does the value of temp1 relate to the average value of temperature?

14. Like vectors, matrices also have norms. The norm kAk of a matrix A describes how far
A is from the zero matrix in a certain sense. Enter the following command into Matlab
norm(P1*P1-P1)

Your answer should be something like ans = 2.0134e-15.

Q2: What does the result say about the relationship between the projection matrix P
and its square P2 ?
15. While the previous approach of approximating the data with a constant function may be
acceptable in some cases, it is clearly not a perfect match. Let us try instead to model the
data as a linear function of yearsi :
ti = b yearsi + c + noise.

55
 If this formula was exact (with no noise), we would have that
   
t1 years1 1
 t2   years2 1   
 b
t= . = ..  c = byears + c1,
  
. ..
 .   . . 
tn yearsn 1

so t would be a vector in the subspace span{years, 1}. However, as before, this is not
exactly true. So, we will need to use an orthogonal projection again.
There is a nice formula for projecting onto a subspace S generated by a span of several
vectors span{b1 , b2 , · · · , bk }, but it requires that the vectors be orthonormal. That is
kbi k = 1 for each i, and bTi bj = 0 for all i 6= j.
One way to produce an orthonormal basis in Matlab is to use the Matlab orth function.
Please, type in the code
m=length(years);
B2=[ones(m,1) years];
Q2=orth(B2);

This creates a matrix B = [1 years], whose first column consists entirely of ones and
whose second column is the vector years, which provides the years of observation. The
new array Q2 is defined to be the result of orth(B2).
The function orth will create an orthonormal basis (defined by the columns of Q2) for the
subspace S. To check that the columns of Q2 form a basis, verify in the command window
that the values of rank(Q2) and rank([Q2,B2]) are both equal to 2.
Variables: Q2
Q3: What kind of matrix is the matrix Q2T Q2? Why?

16. Given an orthonormal basis defined by the columns of Q2 for the subspace S, the formula
for the projection matrix onto the subspace S is simple:

P2 = Q2Q2T .

Create a new cell and in this cell create a variable P2 to hold the value of the matrix
of the orthogonal projection onto the subspace S. Repeat the previous steps to produce
the projection vector temp2=P2*temp and plot this vector in the previous figure window.
This will produce the linear approximation to the given data. Run the following code in
Matlab:
norm(P2*P2-P2}

You should get a number which is close to zero again, such as, 5.6169e-16. There is a
geometric explanation for this fact. The matrices P1, P2 are the projection matrices, and
projecting a vector on a subspace produces a vector belonging to this subspace. Taking a
second projection, thus, will simply leave the vector as is. The projection matrix behaves
as an identity matrices on the corresponding subspace if the basis consisting of the columns
of the matrix Q2 is selected.
Variables: P2, temp2

56
 Figure 10: Approximation of data by a quadratic function.

17. Finally, let us approximate the data by using a quadratic function. Create a new cell
and reproduce the code from from the previous cell but with renamed variables (e.g., B3
instead of B2). Make sure to copy the plotting code and switch P2 to P3.
Aside from changing 2s to 3s, the only other change you’ll need to make is to add a column
to the B3 matrix. This column should be the vector of squares of the elements of the vector
years, which can be produced in Matlab with the expression years.^2 (notice the dot
before the power sign!).
Variables: B3, Q3, P3

18. Plot the initial data and all the approximations above on a new plot (use the command
figure). If correct, your code should produce a figure similar to the one on the Fig. 10.
Observe that the quadratic function (shown by the magenta line) appears to follow the
global tendencies of the temperature graph pretty well.
19. Now, let us make predictions for the future. Create a new cell and run the following code:
futureyears=2016:2116;
futuretemp3=interp1(years,temp3,futureyears,’spline’,’extrap’);
plot(futureyears,futuretemp3,’g-’);

This should create our prediction for an average temperature for the next 100 years. If you
did everything correctly, provided the current tendencies hold, by the year 2116 the average
temperature should be about 3.5C. Note that climate prediction is notoriously difficult,
and it is very likely that there are factors which influence the data which are not accurately
picked up by our simple model. Also, please note that in the past, temperatures tended
to rise faster than any prediction. At the same time, NASA reports that the currently
available models for climate change predict rise in temperatures anywhere between a 2C
and 6C, so our simple model predictions are right in the middle!

57
Project 9: Orthogonal matrices and 3D graphics
Goals: To explore applications of orthogonal matrices to rotation of 3D objects.

To get started: Download the files lab09.m9 , v.mat, and f.mat. The last two files contain
corresponding vertices and faces of a 3D model10 . Save these files in your working directory.

Matlab commands used: function, zeros, sin, cos, if... end, for... end, line,
figure, hold on, hold off, bucky, size, load

What you have to submit: The files rotation.m and lab09.m which you will create during
the lab session.

INTRODUCTION

Recall that the square matrix Q is called orthogonal if QT Q = QQT = I, where I is an

identity matrix of the same size as Q. In other words, this means that QT = Q−1 . Rows
(columns) of an orthogonal matrix generate an orthonormal set of vectors. Namely, if Q =
[q1 , q2 , . . . , qn ], where qj are the columns of the matrix Q, then qj · qk = δjk , and δjk denotes
the Kronecker symbol: δjk = 1, if j = k, and δjk = 0, if j 6= k.
Another property of orthogonal matrices is that x · y = Qx · Qy for any two vectors x and y,
meaning that multiplication by an orthogonal matrix Q does not change the scalar product of
two vectors. Geometrically, this means that a linear transformation T (x) = Qx of the space Rn is
equivalent to the rotation of the space together with a possible reflection along some hyperplane.
Whether a reflection of the space is involved can be seen by looking at the determinant of the
matrix Q. If detQ = 1, then the transformation is a pure rotation. If detQ = −1, then the
transformation is a rotation together with a reflection.
Orthogonal matrices can be used in 3D graphics. Let us consider a three-dimensional object
which we want to show on a computer screen. The obvious difficulty is that the object is three-
dimensional and the screen is only two-dimensional. Thus, we need to generate a projection to
show the object on the screen. One of the easiest ways to do this is to simply drop one of the
coordinates. However, that allows us to view the object only from three different angles which
can make it difficult to understand the real shape of the object. Using orthogonal matrices
allows us to look at the object from many different angles. To accomplish this, we need three
matrices which represent rotations around each of the coordinate axes Ox, Oy, and Oz:
     
1 0 0 cos θy 0 − sin θy cos θz − sin θz 0
Rx =  0 cos θx − sin θx  , Ry =  0 1 0  , Rz =  sin θz cos θz 0  .
0 sin θx cos θx sin θy 0 cos θy 0 0 1

Here θx , θy , and θz are the angles of rotation around the axes 0x, 0y, and 0z correspondingly
in the counterclockwise direction. Multiplying these matrices will result in a combined rotation
around all three axes:
R = Rx Ry Rz .
In this project we will use the matrix R to create a two-dimensional projection of a three-
dimensional object.
9
The printout of the file lab09.m can be found in appendices of this book.
10
The original data for this 3D model can be found here: https://www.thingiverse.com/thing:906692. The
model is licensed under Creative Commons Attribution Share Alike license.

58
 In computer graphics, most objects are not actually smooth, but rather represented by
a polytope. In other words, they are obtained by identifying vertices on the surface of an
object and connecting those vertices with straight lines (edges). Thus, the surface of an object
actually consists of flat faces. In 3D graphics the faces are usually triangular. There is a simple
mathematical reason for this: there always exists a plane passing through any three points in a
space but not necessary through four or more points. If the number of faces is large, then the
object observed from far away appears to be smooth.
Two things are necessary to create a three-dimensional computer model of an object: coor-
dinates of the vertices and the edges showing the connections between the vertices. The former
can be given in a form of an n × 3 array V , where n is the number of vertices in a 3D model.
The latter can be given in several different ways, for instance, by an n × n adjacency matrix E
(hence, a 3D model can be treated as a graph!) or by an m × 3 array containing numbers of
vertices in each face of the model where m is the number of faces. We will look at both of these
descriptions.
Observe that this project is intended as a demonstration of techniques used to create a
projection of a three dimensional object on a computer screen. Of course, Matlab has built in
procedures to work with 3D objects. We will explore these procedures at the end of the project
as well.

TASKS

1. First of all, let us create a user defined function which will return a rotation matrix based
on the user-specified angles θx , θy , and θz which correspond to the rotations around the
axes 0x, 0y, and 0z. Create a new Matlab script and save it as rotation.m in your working
directory. Define the header of this function as:

function rotmat=rotation(theta_x,theta_y,theta_z)

This function takes in three variables theta x,theta y,theta z and returns the 3 × 3
rotation matrix rotmat.
Variables: theta x, theta y, theta z

2. Within the function, create the matrices Rx, Ry, and Rz as defined above by using standard
Matlab syntax and the trigonometric functions sin, cos.
Variables: Rx, Ry, Rz

3. Compute the product of these matrices and assign the result to the output variable rotmat.
Variables: rotmat

4. Save the resulting function in the file rotation.m in your working directory. Now that
the user-defined function is created, we can use it to generate projections.
5. Let us start by creating a projection of a simple object such as a cube. Open the file
m551lab11.m and notice the following code in the first code cell %% Cube:
Vertices=[1 1 1;...
-1 1 1;...
1 -1 1;...
1 1 -1;...
-1 -1 1;...
-1 1 -1;...

59
 1 -1 -1;...
-1 -1 -1];

Edges=zeros(8,8);
Edges(1,2)=1;
Edges(1,3)=1;
Edges(1,4)=1;
Edges(2,5)=1;
Edges(2,6)=1;
Edges(3,5)=1;
Edges(3,7)=1;
Edges(4,6)=1;
Edges(4,7)=1;
Edges(5,8)=1;
Edges(6,8)=1;
Edges(7,8)=1;

Edges=Edges+Edges’;

This code will create the matrix of the vertices of the cube and the corresponding adjacency
matrix.
Variables: Vertices, Edges

6. Assign the following values to the angles of the rotations θx = π/3, θy = π/4, and θz = π/6.
Call the function rotation to generate the rotation matrix with these angles:

rotmat=rotation(theta1,theta2,theta3);

Variables: rotmat
7. Transform the coordinates of the vertices with the rotation matrix.
VertRot=Vertices*rotmat’;

Variables: VertRot
8. Create a new figure window. Moving through the elements of the matrix Edges, draw a
projection of the cube:
figure;
axis equal;
hold on;
for j=1:8
for k=j+1:8
if (Edges(j,k)==1)
line([VertRot(j,1),VertRot(k,1)],[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;
hold off;

The function line draws a line through two points given by their coordinates. The points
can have two or three coordinates. Observe that to generate the projection, we dropped
the last coordinate of the vector VertRot.
Q1: Why does the second for cycle start with j+1?

60
 9. Modify the code above to obtain the projection generated by dropping the second coordi-
nate of the vector VertRot.
10. Let us consider a more complicated object. Matlab has a built in buckyball (bucky)
procedure which produces an array containing the vertices of a buckyball and the adjacency
matrix showing how to connect the vertices:
[Edges2,Vertices2] = bucky;

Variables: Edges2, Vertices2

11. Using the same procedure as above, produce a projection of the buckyball onto the screen.
You can use the same rotation matrix rotmat as before. Find the number of vertices in
the array Vertices2 using the size function.
12. To draw a three dimensional buckyball, use the following code:
figure;
axis equal;
hold on;

for j=1:60
for k=j+1:60
if (Edges2(j,k)==1)
line([Vertices2(j,1),Vertices2(k,1)],[Vertices2(j,2),Vertices2(k,2)],...
[Vertices2(j,3),Vertices2(k,3)]);
end;
end;
end;

hold off;

Observe that you can use the “3D rotation” button in the top panel of the figure window
to rotate the figure. If you use the same button on the previous figure, you will notice that
the generated projection is flat. If you did everything correctly then you will see something
similar to the Fig. 11.
Q2: Why do we use 60 as the end limit for our for cycles?

Figure 11: Buckyball in Matlab window.

61
13. Finally, this can be applied to any three dimensional construction. To illustrate this
consider a model for 3D printing11 . Load the matrices v and f into Matlab from the
files v.mat and f.mat using load command. The array v contains the vertices of the
model, and the array f contains the triangular faces of the model. Each row of the face
array f contains the numbers which identify the vertices in the face. The corresponding
coordinates of the vertices can then be found by referring to the array v.
Variables: v, f

14. Determine the size of the matrix f by using size function. Save the dimensions of this
matrix as mFaces, nFaces. The number of rows mFaces in the array f will give you the
number of faces in the model, while the number of columns nFaces will give you the
number of vertices in each face (it should be equal to three).
Variables: mFaces, nFaces .

Figure 12: 3D model of a fawn.

15. To generate the three dimensional model, use the following code:
figure;
axis equal;
hold on;
for j=1:mFaces
line([v(f(j,1),1),v(f(j,2),1)],[v(f(j,1),2),v(f(j,2),2)],...
[v(f(j,1),3),v(f(j,2),3)]);
line([v(f(j,1),1),v(f(j,3),1)],[v(f(j,1),2),v(f(j,3),2)],...
[v(f(j,1),3),v(f(j,3),3)]);
line([v(f(j,2),1),v(f(j,3),1)],[v(f(j,2),2),v(f(j,3),2)],...
[v(f(j,2),3),v(f(j,3),3)]);
end;
hold off;

If you did everything correctly you will see a figure similar to Fig. 12.

16. Using a similar procedure as above, generate a two dimensional projection of the three
dimensional model. You can use the same rotation matrix rotmat as before. Store the
11
The original data for the model is available from https://www.thingiverse.com/thing:906692/#files

62
 coordinates of the projection in the matrix VertRot.
Variables: VertRot
17. Another way to generate an image like that in Fig. 17 is as follows.
% create rotation matrix
rotmat2 = rotation(-pi/3,0,pi/4);
% rotate vertices
vRot = v*rotmat2’;
% project to xy plane
vPrj = vRot(:,1:2);
% draw
figure;
axis equal;
hold on;
for j=1:mFaces
line([vPrj(f(j,1),1),vPrj(f(j,2),1)],[vPrj(f(j,1),2),vPrj(f(j,2),2)]);
line([vPrj(f(j,1),1),vPrj(f(j,3),1)],[vPrj(f(j,1),2),vPrj(f(j,3),2)]);
line([vPrj(f(j,2),1),vPrj(f(j,3),1)],[vPrj(f(j,2),2),vPrj(f(j,3),2)]);
end;
hold off;

Variables: rotmat2, vRot, vPrj

63
Project 10: Discrete dynamical systems, linear transformations
of the plane, and the Chaos Game
Goals: To use matrix mappings of the plane R2 in the form of a Chaos Game in order to
produce fractals.

To get started: Download the Matlab script lab10.m12 and save it in your working directory.
Matlab commands used: linspace, for... end, rand, randi, plot,
hold on... hold off, cos, sin

What you have to submit: The file lab10.m which you will modify during the lab session.

INTRODUCTION

Many processes can be described by using dynamical systems. The main idea here is that
the state of a system in the current moment may be derived by knowing the state of the system
at the previous moment(s) of time. For instance, if we know that the velocity of a car increases
with the acceleration a = 2 m/s2 and the initial velocity of the car is v0 = 15 m/s, then we can
easily compute the velocity of the car after t seconds by using the formula v = at + v0 . This is
one of the simplest examples of dynamical systems. Motion of celestial bodies and population
dynamics of species are some of the examples of dynamical systems in nature. If the time t is
measured continuously then we are talking about continuous dynamical systems. If the time
is measured discretely (only at certain points), then we are talking about discrete dynamical
systems. For instance, the population of certain species at any moment of time depends on its
population in the previous years. Since offspring for many species is produced only once a year,
we can look at this population dynamics as a discrete dynamical system with time measured in
years.
In this project we will explore the connection between fractals and matrix (linear) mappings
in the form of the so-called Chaos Game. The Chaos Game is an algorithm first proposed by
M. Barnsley in 1998 in his book “Fractals everywhere”.
The basic idea of the Chaos Game is simple. Pick n points on the plane (say, vertices of a
regular n-gon), and a number r, 0 < r < 1. Pick a random point inside the polygon. Then repeat
the following steps: pick a polygon vertex at random and draw the next point at a fraction r
of the distance between the current point and the chosen vertex of the polygon. Repeat the
process with the new point.
This process can be written in the form of the following linear mapping. Let x = (x1 , x2 )
be the current point, and let v = (v1 , v2 ) be the selected vertex of the polygon. Then the next
point can be generated by using the linear mapping T :
 
T11 T12
T (x) = (x − v) + v.
T21 T22

Here T (x) is the next point. Repeated many times, this process sometimes (but not always)
produces a fractal. The mapping T can change from iteration to iteration or can be picked at
random with different probabilities (we will look at the examples of this type later). In the tasks
below, we will look at some limiting sets of points which can be obtained by this procedure.

12
The printout of the file lab10.m can be found in the appendices of this book.

64
 TASKS

1. Let us start simple by creating an image of the Sierpinski Triangle using the Chaos Game.
Locate the cell titled %% Sierpinski Triangle. In this cell, define the vertices of an
equilateral triangle by using the following commands
clear;
t=linspace(0,2*pi,4);
t(4)=[];
v=[cos(t); sin(t)];

This code creates a set of vertices for equilateral triangle. Each vertex is a distance of 1
unit away from the origin and the first vertex is located on the Ox axis.
Variables: t,v
Q1: What does the command t(4)=[] do?

2. Next, define the starting point and the matrix of the linear transformation:
T=[1/2 0; 0 1/2];
x(:,1)=[rand-0.5; rand-0.5];

The linear transformation with the matrix T is equivalent to multiplying the vector by 1/2,
and the starting point x(:,1) is picked at random.
Variables: T,x

3. Plot the vertices of the triangle on the plane and select the number of iterations of the
process:
plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=10000;

All the points which will be obtained during the iterative process will be stored in the
2 × N array x where N is the number of iterations. After computing all the iterations we
need, we will plot all the points from the array x using the command plot. Observe that
while it is possible to immediately plot the point we obtain at each step, multiple calls to
the plot function take a significant amount of computational time and, thus, doing so is
impractical.
Variables: Num
4. The following code creates the iterated points and plots them:
for j=1:Num
k=randi(3);
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

In this code, we select an integer number k from 1 to 3 at random by using the command
randi. We then do a linear transformation with the matrix T, the selected vertex v(:,k),
and the point x(:,j) obtained on the previous step. Running this code with a large
number of points may take some time. If you did everything correctly, you should obtain

65
 Figure 13: The Sierpinski triangle

an image similar to Fig. 13. The resulting picture approximates a fractal called the
Sierpinski triangle.
There is a different procedure to generate this fractal starting with the interior of an
equilateral triangle. At each step we will connect the mid-points of the sides of the triangle
by straight lines. This subdivides the triangle into four triangles with the sides twice
smaller than the original. Now we will throw out the middle triangle. Then we repeat this
process with each of the leftover triangles, and so on. Doing this infinitely many times will
produce the Sierpinski triangle as well.
Variables: x

5. Let us try some variations of the previous procedure. First of all, let us change the ratio in
which the segment is divided. Locate the code cell titled %% Sierpinski triangle with
a change of ratio and run the previous code with T=[1/3 0; 0 1/3]. Don’t forget to
clear the variables first (use the command clear)!
6. We can also add rotation to each of the iterations. Locate the code cell titled %% Sierpinski
triangle with rotation. Use the original algorithm for Sierpinsky triangle but change
the matrix T to the following::
theta=pi/18;
T=1/2*[cos(theta) -sin(theta); sin(theta) cos(theta)];

and run the procedure again. Observe that the matrix T is equivalent to two transforma-
tions: the shrinking of the plane by 1/2 and the counterclockwise rotation of the plane
around the origin with the angle of rotation theta.
7. Next, let us look at the fractal generated starting from four vertices of a square. Locate
the code cell titled %% Square and run the following code:
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

Now repeat the iterative procedure allowing the vertex at each step to be randomly selected
from the four vertices of the square. Generate the points and plot them on a new figure.
Observe that no fractal has been created in this case.

66
 8. Locate the code cell titled %% Square with different ratio. Change the ratio in which
the segment is divided to 1/3 and run the code again (don’t forget to clear the variables!)
with the modified matrix T:
T=[1/3 0; 0 1/3];

Q2: Does the code generate a fractal this time?

9. Let us try to do more sophisticated things. Locate the code cell titled %% Square with
vertex preference (the new vertex cannot be the same as the previous) which
contains the following code:
clear;
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=exp(1i*t);
x(:,1)=[rand-0.5; rand-0.5];
plot(real(v),imag(v),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=5000;
k1=0;
for j=1:Num
k=randi(3);
if (k>=k1)
k=k+1;
end;
w=[real(v(k));imag(v(k))];
x(:,j+1)=T*(x(:,j)-w)+w;
k1=k;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

This code repeats the process for the square except that it does not allow the same vertex
to be selected twice in a row.
10. Try the next code cell titled %% Square with vertex preference (the new vertex cannot
be opposite of the previous)
%% Square with vertex preference (new vertex cannot be opposite of the previous):
clear;
T=[1/2 0; 0 1/2];
t=linspace(0,2*pi,5);
t(5)=[];
v=exp(1i*t);
x(:,1)=[rand-0.5; rand-0.5];
plot(real(v),imag(v),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;
Num=5000;
w=[real(v(1));imag(v(1))];
k1=0;

67
 for j=1:Num
k=randi(4)
if (k~=k1+2)&&(k1~=k+2)
w=[real(v(k));imag(v(k))];
x(:,j+1)=T*(x(:,j)-w)+w;
k1=k;
else
x(:,j+1)=x(:,j);
end;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

This code repeats the process above except that it does not allow for the new vertex to lie
opposite the old one.
11. Run the next code cell titled %% Barnsley fern. The code in this cell creates the so-called
Barnsley fern which is a fractal with striking resemblance to a fern leaf (a lot of plants
and animals in nature use fractals to their advantage!):
%% Barnsley fern
clear;

T1=[0.85,0.04;-0.04,0.85];
T2=[-0.15,0.28;0.26,0.24];
T3=[0.2,-0.26;0.23,0.22];
T4=[0,0;0,0.16];
Q1=[0;1.64];
Q2=[-0.028;1.05];
Q3=[0;1.6];
Q4=[0,0];
P1=0.85;
P2=0.07;
P3=0.07;
P4=0.01;

Num=15000;
x(:,1)=rand(2,1);
plot(x(1,1),x(2,:),’b.’)
axis equal;
hold on;
for j=1:Num
r=rand;
if r<=P1
x(:,j+1)=T1*x(:,j)+Q1;
elseif r<=P1+P2
x(:,j+1)=T2*x(:,j)+Q2;
elseif r<=P1+P2+P3
x(:,j+1)=T3*x(:,j)+Q3;
else
x(:,j+1)=T4*x(:,j)+Q4;
end
end

plot(x(1,:),x(2,:),’b.’);

68
 Figure 14: Barnsley Fern

Figure 15: Fractals

hold off;

This code alternates between four different linear mappings given by y = Tj ∗ x + Qj ,

j = 1, 2, 3, 4, where the matrices Tj and the vectors Qj are given in the code. The
probabilities of each mapping being selected are not equal as well and are given by the
numbers P1, P2, P3, and P4 in the code. If you did everything correctly, you should see
the image similar to the one in Fig. 14.
12. Modify the initial code for the Sierpinski triangle to create a fractal based on a regular
hexagon. Take the matrix T to be
T=[1/4 0; 0 1/4];

If you did everything correctly, you should see the figure similar to the first image in Fig.
15.
Q3: Experiment with the ratio. What is the largest ratio you can find such that
the little hexagons don’t touch (two digits after the decimal point are sufficient)?

13. Finally, create another Chaos Game. Use the matrix T=[2/5 0; 0 2/5] for the linear
mapping and the vertices of a regular pentagon with the restriction that each new vertex
cannot be the same as the previous one. Plot the result in a new figure window. If you
did everything correctly, you should see the figure similar to the second image in Fig. 15.

69
Project 11: Projections, eigenvectors, Principal Component Anal-
ysis, and face recognition algorithms
Goals: To use eigenvalues and eigenvectors of a matrix to create a simple face recognition
algorithm.

To get started:

• Download the file lab11.m and put it in your working directory13 .

• Download the archive file database.zip and unpack it into your working directory. This
will produce a folder “database”, which contains 33 image files in the format .pgm 14 .

Matlab commands used: imread, size, reshape, num2str, mean, uint8, double,
ones, eig, diag

What you have to submit: The file lab11.m which you will modify during the lab session.

INTRODUCTION

In this project we will apply principal component analysis (PCA) to create a simple face
recognition algorithm. The PCA algorithm works by identifying patterns in the data which
allows one to reduce the dimension of the dataset. We will use the following steps to apply the
PCA algorithm to face recognition:

• Download the database of N images (in our case, N = 30). All images must have the
same dimensions m × n.

• Input the images person1.pgm-person30.pgm into Matlab as arrays. Reshape each of the
arrays as a column vector Xi , i = 1, ..., N .

• Compute the mean of all images Xmean = N1 N

P
i=1 Xi .

• Subtract the mean image from each of the vectors Xi : X̃i = Xi − Xmean and create a
matrix P, the ith column of which is equal to the vector X̃i . The matrix P has 30 columns
and mn rows where m, n are the dimensions of the images.

• We are interested in the eigenvalues and eigenvectors of the matrix PPT . However, the
matrix PPT is usually very large. For instance, if our database consists of the images with
dimensions m × n = 100 × 100, then the matrix PPT has the dimensions 10000 × 10000,
and computation of eigenvalues and eigenvectors becomes unfeasible. Instead, we will use
the fact that only at most N − 1 eigenvalues of the matrix PPT are non-zero (why?),
and those non-zero eigenvalues are the same as non-zero eigenvalues of the matrix PT P
(why?). The matrix PT P has the dimensions N × N which are usually much smaller than
the dimensions of the matrix PPT . For instance, if there are 30 images in the database
(like in our case), then the dimensions of PT P are only 30 × 30. Thus, it is much more
efficient to solve the eigenvalue problem for the matrix PT P.
13
The printout of the file lab11.m can be found in appendices of this book.
14
These files represent a subset of the database of faces from AT&T Laboratories Cambridge www.cl.cam.ac.
uk/research/dtg/attarchive/facedatabase.html.

70
• If ui is an eigenvector of the matrix PT P corresponding to a non-zero eigenvalue λi , then
vi = Pui is the eigenvector of the matrix PPT corresponding to λi . The eigenvectors of
the matrix PPT constitute the set of components of the data with the largest variance
between them (which are called principal components).

• Compute the projections of your sample data onto the linear subspace generated by the
vectors vi . The power of the algorithm is in the dimension reduction. Our initial images
of faces “live” in the mn-dimensional real space. For instance, if the images are 100 × 100
pixels, then the dimension of the space is equal to 1002 = 10000. If we would try to
compare these images to each other pixel by pixel, we would need to compare 10000
pixels. However, not all of these pixels are significant. Principal Component Analysis
allows us to identify the important information about the image and compare only that
information. For instance, if we are working with a database of 30 images, then we will
need to compare only 29 numbers. That is a dimensional reduction from 10000 to only
29! The PCA algorithm is a work horse behind many modern techniques in data mining,
including face recognition algorithms.

TASKS

1. Open the file lab11.m and run the first code cell titled %% Input database files into
the Matlab:
%%Input database files into Matlab
clear;
Database_Size=30;
%Reading images from the database. The image files should be located in the
%subfolder ‘‘database"
for j=1:Database_Size
image_read=imread([’database\person’ num2str(j) ’.pgm’]);
[m,n]=size(image_read);
P(:,j)=reshape(image_read,m*n,1);
end;

This code will input the database images into Matlab, reshape each image from the rect-
angular array to a column vector, and create the matrix P. Observe the use of the string
function num2str in the imread command and the use of the reshape function to change
the dimensions of the arrays containing the image data. Instead of the rectangular arrays
with the elements corresponding to the pixel colors we now have a set of long vectors where
columns of each image have been “stacked” on top of each other.
Variables: Database Size, m, n, P

2. Compute the mean image by applying the mean function to the the matrix P row-wise.
Display the mean image by using the function imshow.
%% Computing and displaying the mean face
mean_face=mean(P,2);
imshow(uint8(reshape(mean_face,m,n)))

Variables: mean face

3. Subtract the mean of the images from each of the columns of the matrix P:

71
 %% Subtract the mean face
P=double(P);
P=P-mean_face*ones(1,Database_Size);

Notice the conversion to the double type of the array P.

4. Compute the covariance matrix PT P and compute the eigenvalues and the eigenvectors
of this matrix PT P by using the built in function eig. The function eig returns two
arrays, one of which has eigenvalues on the main diagonal, and the other has eigenvectors
as columns. Observe that one of the eigenvalues is very close to zero.
%% Compute the covariance matrix of the set and its eigenvalues and
eigenvectors
[Vectors,Values]=eig(P’*P);
EigenVectors=P*Vectors;

Variables: Vectors, Values, EigenVectors

Q1: Why does the matrix PT P have a zero eigenvalue?

5. Display the set of eigenfaces on the same figure by running the code cell %% Display the
set of eigenfaces:
%% Display the set of eigenvaces
for j=2:Database_Size;
if j==2
EigenFaces=reshape(EigenVectors(:,j)+mean_face,m,n);
else
EigenFaces=[EigenFaces reshape(EigenVectors(:,j)+mean_face,m,n)];
end;
end
EigenFaces=uint8(EigenFaces);
figure;
imshow(EigenFaces);

If you did everything correctly, you will see an image containing 29 eigenfaces, the first
five of which are shown in Fig. 16. Observe that the mean face has been added back to
the eigenfaces.
Variables: EigenFaces

6. The matrix PPT is symmetric. It follows from the theorems of linear algebra that the
eigenvectors corresponding to the distinct eigenvalues are orthogonal. Write code to verify
the last statement. You will need to create the matrix
Products=EigenVectors’*EigenVectors
and observe that it is a diagonal matrix. This matrix contains all possible dot products
of the eigenvectors of the matrix PPT . The fact that the eigenvectors are orthogonal will
be used in the following sections to find orthogonal projections.
Variables: Products
Q2: Why is the matrix PPT symmetric?

7. In this project we will use the obtained eigenfaces to perform several operations. Our
first task will be to “unmask” an altered face (sunglasses have been digitally added to the
image). The main idea behind this code is in computing the projection of the vector of data
(obtained from the altered image) onto the subspace spanned by the eigenvectors. This

72
 Figure 16: The first five eigenfaces

projection is then displayed on the screen. Now run the code cell titled %% Recognition
of an altered image (sunglasses):
%% Recognition of an altered image (sunglasses)
image_read=imread([’database\person30altered1.pgm’]);
U=reshape(image_read,m*n,1);
NormsEigenVectors=diag(Products);
W=(EigenVectors’*(double(U)-mean_face));
W=W./NormsEigenVectors;
U_approx=EigenVectors*W+mean_face;
image_approx=uint8(reshape(U_approx,m,n));
figure;
imshow([image_read,image_approx])

In this code, we start by uploading the image person30altered1.pgm into Matlab. The
image is then reshaped as a long vector similarly to the way it was done before. Then we
compute the projection of the new vector with the subtracted vector mean face onto the
space spanned by the set of the eigenvectors and add the vector mean face back. Finally,
the reconstructed image is displayed. Observe that we get a very good approximation of
the initial image without sunglasses.
Variables: image read, U, NormsEigenVectors, W, U approx

8. Modify the code above to unmask the image person30altered2.pgm.

9. Now let us compute the approximation of the face which is not in the database. To do this,
modify the code above again to approximate the image person31.pgm. Observe, that the
results of the last two approximations are somewhat worse than the very first one. This
can be explained by the fact that there is an insufficient number of images in our database
which do not span a wide enough variety of faces.

73
Project 12: Matrix eigenvalues and the Google’s PageRank algo-
rithm
Goals: To apply matrix eigenvalues and eigenvectors to ranking of webpages in the World Wide
Web.

To get started:

• Download the file lab12.m and put it in your working directory15 .

• Download the file AdjMatrix.mat which contains the adjacency matrix of a so-called “wiki-
vote” network with 8297 nodes 16 [9].

Matlab commands used: load, size, numel, nnz, for... end, gplot

What you have to submit: The file lab12.m which you will modify during the lab session.

INTRODUCTION

According to social polls, the majority of users only look at the first few results of the online
search and very few users look past the first page of results. Hence, it is crucially important to
rank the pages in the “right” order so that the most respectable and relevant results will come
first. The simplest way to determine the rank of a webpage in a network is to look at how many
times it has been referred to by other webpages. This simple ranking method leaves a lot to be
desired. In particular, it can be easily manipulated by referring to a certain webpage from a lot
of “junk” webpages. The quality of the webpages referring to the page we are trying to rank
should matter too. This is the main idea behind the Google PageRank algorithm.
The Google PageRank algorithm is the oldest algorithm used by Google to rank the web
pages which are preranked offline. The PageRank scores of the webpages are recomputed each
time Google crawls the web. Let us look at the theory behind the algorithm. As it turns out it,
it is based on the theorems of linear algebra!
The main assumption of the algorithm is that if you are located on any webpage then with
equal probability you can follow any of the hyperlinks from that page to another page. This
allows to represent a webpage network as a directed graph with the webpages being the nodes,
and the edges being the hyperlinks between the webpages. The adjacency matrix of such a
network is built in the following way: the (i, j)th element of this matrix is equal to 1 if there
is a hyperlink from the webpage i to the webpage j and is equal to 0 otherwise. Then the row
sums of this matrix will represent numbers of hyperlinks from each webpage and the column
sums will represent numbers of times each webpage has been referred to by other webpages.
Even further, we can generate a matrix of probabilities S such that the (i, j)th element of
this matrix is equal to the probability of traveling from ith webpage to jth webpage in the
network. This probability is equal to zero if there is no hyperlink from ith page to jth page and
is equal to 1/Ni if there is a hyperlink from ith page to jth page, where Ni is the total number of
hyperlinks from ith page. For instance, consider a sample network of only four webpages shown
on the Fig. 17. The matrix S for this network can be written as:
15
The printout of the file lab12.m is given in the appendices of this book.
16
The original network data is available here: https://snap.stanford.edu/data/.

74
 1 2

4 3
Figure 17: Sample network of four webpages

 
0 1/3 1/3 1/3
 1/2 0 0 1/2 
S=
 0 1/2 0 1/2 
 (1)
0 0 0 0

There are several issues which make working with the matrix S inconvenient. First of all,
there are webpages that do not have any hyperlinks - the so-called “dangling nodes” (such as
the node 4 in Fig. 17). These nodes will correspond to zero rows of the matrix S. Moreover, the
webpages in the network may not be connected to each other and the graph of the network may
consist of several disconnected components. These possibilities lead to undesirable properties of
the matrix S which make computations with it complicated and not even always possible.
The problem of the dangling nodes can be solved by assigning all elements of the matrix S in
the rows corresponding to the dangling nodes equal probabilities 1/N , where N is the number
of the nodes in the network. This can be understood in the following way: if we are at the
dangling node we can with equal probability jump to any other page in the network. To solve
the potential disconnectedness problem, we assume that a user can follow hyperlinks on any
page with a probability 1 − α and can jump (or “teleport”) to any other page in the network
with a probability α. The number α is called a damping factor. The value of α = 0.15 is usually
taken in practical applications. The “teleport” surfing of the network can be interpreted as a
user manually typing the webpage address in the browser or using a saved hyperlink from their
bookmarks to move from one page onto another. The introduction of the damping factor allows
us to obtain the Google matrix G in the form:

G = (1 − α)S + αE,

where E is a matrix with all the elements equal to 1/N , where N is a number of webpages in
the network.
The matrix G has nice properties. In particular, it has only positive entries and all of its
rows sum up to 1. In mathematical language, this matrix is stochastic and irreducible (you can
look up the precise definitions of these terms if you are interested). The matrix G satisfies the
following Perron-Frobenius theorem:

Theorem 1 (Perron-Frobenius) Every square matrix with positive entries has a unique unit
eigenvector with all positive entries. The eigenvalue corresponding to this eigenvector is real and
positive. Moreover, this eigenvalue is simple and is the largest in absolute value among all the
eigenvalues of this matrix.

Let us apply this theorem to the matrix G. First of all, observe that the row sums of the

75
matrix G are equal to 1. Consider the vector v1 = (1, 1, ..., 1)T /N . It is easy to see that

Gv1 = v1 .

But then it follows that v1 is the unique eigenvector with all positive components, and, therefore,
by the Perron-Frobenius theorem, λ1 = 1 is the largest eigenvalue!
We are interested in the left eigenvector for the eigenvalue λ1 = 1:

uT1 G = uT1 .

Again, by the Perron-Frobenius theorem, the vector u1 is the unique unit eigenvector with all
positive components corresponding to the largest in absolute value eigenvalue λ1 = 1. We will
use the components of this vector for the ranking of webpages in the network.
Let us look at the justification behind this algorithm. We have already established that the
vector u1 exists. Consider the following iterative process. Assume that at the beginning a user
can be on any webpage in the network with equal probability:

w0 = (1/N, 1/N, ..., 1/N ).

After 1 step (one move from one webpage to another using hyperlinks or teleporting), the
probability vector of being on the ith webpage is determined by the ith component of the vector

w1 = w0 G.

After two moves the vector of probabilities becomes

w2 = w1 G = w0 G2 ,

and so on.
We hope that after a large number of steps n, the vector wn = w0 Gn starts approaching
some kind of limit vector w∗ , wn → w∗ . It turns out that due to the properties of the matrix G
this limit vector w∗ indeed exists and it is exactly the eigenvector corresponding to the largest
eigenvalue, namely, λ1 = 1. Moreover, numerical computation of matrix eigenvalues is actually
based on taking the powers of the matrix (it is called the Power method) and not on solving the
characteristic equation!
Let us assume that the vector w∗ is a non-negative vector whose entries sum to 1. Then the
components of this vector represent the probabilities of being on each webpage in the network
after a very large number of moves along the hyperlinks. Thus, it is perfectly reasonable to take
these probabilities as ranking of the webpages in the network.

TASKS

1. Open the file lab12.m. In the code cell titled %%Load the network data load the data
from the file AdjMatrix.mat into Matlab by using the load command. Save the resulting
matrix as AdjMatrix. Observe that the adjacency matrices of real networks are likely to
be very large (may contain millions of nodes or more) and sparse. Check the sparsity of
the matrix AdjMatrix using the functions numel and nnz. Denote the ratio of non-zero
elements as nnzAdjMatrix. If you did everything correctly you should obtain that only
0.15% of the elements of the matrix AdjMatrix are non-zero.
Variables: AdjMatrix, nnzAdjMatrix

76
2. Check the dimensions of the matrix AdjMatrix using the size function. Save the dimen-
sions as new variables m and n.
Variables: m, n

3. Observe that while the network described by the matrix AdjMatrix is not large at all from
the viewpoint of practical applications, computations with this matrix may still take a
noticeable amount of time. To save time, we will cut a subset out of this network and use
it to illustrate the Google PageRank algorithm. Introduce a new variable NumNetwork and
set its value to 500. Then cut a submatrix AdjMatrixSmall out of the matrix AdjMatrix
and plot the graph represented by the matrix AdjMatrixSmall by running the following
code cell:
%% Display a small amount of network
NumNetwork=500;
AdjMatrixSmall=AdjMatrix(1:NumNetwork,1:NumNetwork);
for j=1:NumNetwork
coordinates(j,1)=NumNetwork*rand;
coordinates(j,2)=NumNetwork*rand;
end;
gplot(AdjMatrixSmall,coordinates,’k-*’);

This will plot the subgraph of the first 500 nodes in the network with random locations
of the nodes. Notice the use of the function gplot to produce this graph. Observe
that Matlab has special functions graph and digraph for working with graphs, but those
functions are a part of the special package “Graph and Network Algorithms” which may
not be immediately available. Simpler methods, as shown above, will be sufficient for our
purposes.
Variables: AdjMatrixSmall, coordinates, NumNetwork

4. Set the parameter α = 0.15. Introduce the vector w0 =(1,1,...,1)/NumNetwork, and

compute the consequent vectors w1 = w0 G, w2 = w1 G, w3 = w2 G, w5 = w0 G5 , w10 =
w0 G10 . Compute the difference δw = w10 − w5 . Observe that the sequence wn converges
to a certain limit vector w∗ very fast.
Variables: w0, w1, w2, w3, w5, w10, deltaw

5. Compute the eigenvalues and the left and the right eigenvectors of the matrix G using the
function eig. Observe that the right eigenvector corresponding to the eigenvalue λ1 = 1
is proportional to the vector v1 = (1, 1, ..., 1). To compute the left eigenvectors, use the
function eig on the matrix G’. Select the left eigenvector corresponding to the eigenvalue
λ1 = 1 and denote it as u1.
Variables: u1
6. Observe that by default the vector u1 is not scaled to have all positive components (even
though all the components of the vector u1 will have the same sign). Normalize this vector
by using the code:
u1=abs(u1)/norm(u1,1);

This will create a vector with all positive components whose entries sum to 1 (called a
probability vector).

7. Use the function max to select the maximal element and its index in the array.
Variables: MaxRank, PageMaxRank

77
8. Find out whether the highest ranking webpage is the same as the page with the most
hyperlinks pointed to it. To do so, create the vector of column sums of the matrix G and
save it as MostLinks. Use the function max again to select the element with the maximal
number of links.
Variables: MostLinks, MaxLinks, PageMaxLinks

9. Compare if MaxRank and MaxLinks are the same.

Q1: What is the number of hyperlinks pointing to the webpage MaxRank?
(look at the vector MostLinks to find out).

78
Project 13: Social networks, clustering, and eigenvalue problems
Goals: To apply linear algebra techniques to data clustering.

To get started:

• Download the file lab13.m and put it in your working directory17 .

• Download the file social.mat which contains the adjacency matrix of a simulated social
network18 .

Matlab commands used: sum, diag, eig, for...end, if... then... end, load,
clear, spy, eye

What you have to submit: The file lab13.m which you will modify during the lab session.

INTRODUCTION

Many practical applications deal with large amounts of data. One of the standard problems
in data science is to group this data in clusters containing similar items. In many cases, clustering
can be easily accomplished by the human brain (take a look at the attached pictures).

In general, creating an algorithm which allows us to separate the data into clusters is a difficult
task. The data in many cases is multidimensional and cannot be visualized easily. Even deciding
how many clusters to select is not necessary an easy task. There are currently several widely used
algorithms which allow for the separation of the data points by similarity. All these algorithms
have their own advantages and disadvantages.
In this project, we will use the spectral clustering algorithm based on the Fiedler’s vector.
We will look at the graph representing an individual’s profile on a social network. The profile
has a total of 351 “friends”. Some of these friends know each other and are friends with each
other as well. This friendship network can be represented as a graph with nodes being friends
and edges showing the friendship connection between the individuals. The goal of clustering is
to separate this graph into components which are tightly connected within themselves and have
fewer connections to the outside (the so-called communities). The method explored here is based
on the theorem by M. Fiedler, who has shown that for a connected graph (meaning that there is
a path between any two vertices) the eigenvector corresponding to the second smallest eigenvalue
of the Laplacian of the graph allows the graph to be split into two maximally intraconnected
components.

17
The printout of the file lab13.m can be found in the appendices of this book.
18
This data has been artificially created.

79
 Figure 18: An example of a simple graph

TASKS

1. We will start by considering an example of a simple graph: Create a new code cell with
the title %% Small graph. Type in the adjacency matrix of this graph into Matlab and
save it in a variable AdjMatrix.
Variables: AdjMatrix

2. Find the row sums of the matrix AdjMatrix (use the sum function) and save it as RowSums.
Variables: RowSums
3. Compute the Laplacian of the graph using the following command:
LaplaceGraph=diag(RowSums)-AdjMatrix;

Observe that the diag(RowSums) command creates a diagonal matrix with the vector
RowSums on the main diagonal and zeros everywhere else. Observe also that the ma-
trix LaplaceGraph is a symmetric positive semi-definite singular matrix. Recall that
a matrix A is positive semi-definite if xAxT ≥ 0 for all column vectors x. To check
that it is singular, multiply the matrix LaplaceGraph on the right by the column vector
[1, 1, 1, 1, 1, 1, 1, 1, 1]T .
Variables: LaplaceGraph

4. Find the eigenvalues and eigenvectors of the matrix LaplaceGraph using the eig function:
[V,D]=eig(LaplaceGraph);

Here the matrix V contains all the right eigenvectors of the matrix LaplaceGraph and the
matrix D is a diagonal matrix with eigenvalues of the matrix LaplaceGraph on the main
diagonal. Since the Laplacian is a symmetric graph, its eigenvalues are real. To simplify
our task, let us sort the eigenvalues from smallest to largest. We will need to sort the
eigenvectors along with them. Here is how to do that.
% sort eigenvalues and eigenvectors
[d,ind] = sort(diag(D));
D = D(ind,ind);
V = V(:,ind);

80
 Variables: V, D

5. View the elements of the matrices V and D using the “Workspace” window of the Matlab
environment. Observe that one of the eigenvalues of the matrix LaplaceGraph is zero (this
is due to the fact that the matrix LaplaceGraph is singular). The eigenvector for this
eigenvalue is proportional to the vector [1, 1, 1, 1, 1, 1, 1, 1, 1]T . All other eigenvalues
of the matrix LaplaceGraph are positive. This follows from the fact that the matrix
LaplaceGraph is positive semi-definite and the graph corresponding to this Laplacian is
connected. Find the smallest positive eigenvalue of the matrix LaplaceGraph (since we
sorted the eigenvalues, it is the second diagonal element of the matrix D).
6. Identify the eigenvector corresponding to the second smallest eigenvalue:
V2=V(:,2);

In the next step, we will use the entries of V2 to group the elements. To make sure we do
this consistently, let us make sure that V2 is the unit eigenvector with positive first entry:
% make sure the first entry is positive
if V2(1) < 0
V2 = -V2;
end

Variables: V2
7. Separate the elements of the eigenvector V2 onto positive and negative:
pos=[];
neg=[];
for j=1:9
if V2(j)>0
pos=[pos,j];
else
neg=[neg,j];
end;
end;

If everything is done correctly, the arrays pos and neg will contain correspondingly 1, 2, 3, 4
and 5, 6, 7, 8, 9 which would be an intuitive clustering choice.
Variables: pos, neg

8. Now let us apply this technique to a larger graph when the results cannot be visualized so
easily. To do this, load the matrix Social which is an adjacency matrix of a network of
351 Facebook friends. The (i, j)th element is equal to 1 if the ith and jth friends are also
friends with each other and is equal to 0 otherwise:
%% Load the data
clear;
load(’social.mat’,’Social’);
spy(Social);

The command spy(Social) will plot the sparsity pattern of the adjacency matrix Social
on a figure, putting blue dots for non-zero elements of the matrix (you should see something
similar to the first image in Fig. 19). Observe that it is difficult to distinguish any kind
of underlying pattern in this matrix. Our goal will be to cluster this set of data into two

81
 Figure 19: Adjacency matrices of the unsorted and sorted data.

maximally intraconnected groups.

Variables: Social

9. To do this, repeat the procedure we performed on a smaller matrix. Define the matrices
DiagSocial and LaplaceSocial which are correspondingly the vector of row sums of the
matrix Social and the Laplacian of this matrix.
Variables: DiagSocial, LaplaceSocial

10. Compute the eigenvalues and the eigenvectors:

[V,D]=eig(LaplaceSocial);

Variables: V,D

11. As before, identify the second smallest eigenvalue and the corresponding eigenvector V2
and sort the vertices of the graph by the positive and negative components of the vector
V2. Make the necessary adjustment to ensure that V2(1)>0.
Variables: V2
12. Permute the adjacency matrix Social using the permutation generated by the lists pos,
neg:
order=[pos,neg];
[m,n]=size(Social);
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;
SocialOrdered=P*Social*P’

Variables: order, P, SocialOrdered

Q1: What does the command P*Social*P’ do to the rows and columns of the matrix
Social?

82
13. Plot the matrix SocialOrdered using the function spy again. If you did everything cor-
rectly, you will see the matrix separating into two denser connected subsets similarly to
what is pictured in the second image in Fig. 19.
14. What if we want more clusters? There are two ways to proceed. First, we can explore the
eigenvector V3 corresponding to the third smallest eigenvalue. We will group the nodes
in the four clusters: the ones which have positive components in both eigenvectors V2, V3
(++ group), the ones which have a positive component in V2 and a negative component
in V3 (+−), the ones which have a negative component in V2 and a positive component
in V3 (−+), and, finally, the ones with negative components in both vectors (−−). This
generates 4 distinct groups and can be accomplished by the code in the following code cell:
%% Cluster in 4 groups
V3=V(:,3);
if V3(1) < 0
V3 = -V3;
end
pp=[];
pn=[];
np=[];
nn=[];
for j=1:m
if (V2(j)>0)
if (V3(j)>0)
pp=[pp,j];
else
pn=[pn,j];
end;
else
if (V3(j)>0)
np=[np,j];
else
nn=[nn,j];
end;
end;
end;
order=[pp,pn,np,nn];
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;
SocialOrdered=P*Social*P’
figure;
spy(SocialOrdered)

15. An alternative way to obtain more clusters is to use the Fiedler vector procedure iteratively,
meaning that we will apply it again to the clusters obtained in the previous step. This
can be accomplished by running the following code:
%% Second order of Fiedler
SocialPos=Social(pos,pos);
SocialNeg=Social(neg,neg);
rowsumpos=sum(SocialPos,2);

83
 DiagSocialPos=diag(rowsumpos);
LaplaceSocialPos=DiagSocialPos-SocialPos;
[VPos,DPos]=eig(LaplaceSocialPos);
V2Pos=VPos(:,2);
[mpos,npos]=size(V2Pos);
posp=[];
posn=[];
for j=1:mpos
if (V2Pos(j)>0)
posp=[posp,pos(j)];
else
posn=[posn,pos(j)];
end;
end;
rowsumneg=sum(SocialNeg,2);
DiagSocialNeg=diag(rowsumneg);
LaplaceSocialNeg=DiagSocialNeg-SocialNeg;
[VNeg,DNeg]=eig(LaplaceSocialNeg);
V2Neg=VNeg(:,2);
[mneg,nneg]=size(V2Neg);
negp=[];
negn=[];
for j=1:mneg
if (V2Neg(j)>0)
negp=[negp,neg(j)];
else
negn=[negn,neg(j)];
end;
end;
ordergen=[posp,posn,negp,negn];
iden=eye(m,m);
for j=1:351;
for k=1:351;
P(j,k)=iden(ordergen(j),k);
end;
end;
SocialOrderedGen=P*Social*P’
figure;
spy(SocialOrderedGen)

Observe that the densely connected components obtained by the last two methods might
be different. In the context of a social network, the dense clusters may represent groups of
people who have common connections in real life. For instance, your family, your friends
from high school, your colleagues, etc.

84
Project 14: Singular Value Decomposition and image compres-
sion
Goals: In this project we will discuss a singular value decomposition of a matrix and its
applications to image compression and noise filtering.

To get started:

• Download the file lab14.m and put it in your working directory19 .

• Download the files einstein.jpg20 and checkers.pgm21 and put them in your working
directory.

Matlab commands used: clear, linspace, subplot, quiver, plot, hold on... hold
off, svd, axis, title, imread, imshow, size, min, for... end, cos, sin, ’,
rand, double, ones

What you have to submit: The file lab14.m which you will modify during the lab session.

INTRODUCTION

Any m × n matrix A possesses a singular value decomposition of the form:

A = UΣVT ,

where U is an orthogonal m × m matrix satisfying the condition UT U = Im , Σ is an m × n

rectangular diagonal matrix with nonnegative values σ1 , σ2 , . . . , σmin(m,n) on the main diagonal,
and V is an orthogonal n × n matrix satisfying the condition VT V = In . The nonnegative
numbers σ1 , σ2 , . . . , σmin(m,n) are called the singular values of the matrix A. They are arranged
in the decreasing order: σ1 ≥ σ2 ≥ . . . ≥ σmin(m,n) ≥ 0. Observe that the singular value
decomposition of a matrix is not unique.
Singular value decomposition is important for implementation of many numerical algorithms
in linear algebra. In this project we will look at the geometric meaning of singular value decom-
position and two applications related to image processing and noise filtering. Observe that more
efficient image compression methods exist. The examples in this project are used mainly to show
the reduction in the amount of data (so-called dimension reduction) which can be accomplished
with singular value decomposition. Dimension reduction is an important tool in many practical
applications dealing with large amounts of data, such as statistics, data science, and machine
learning.

TASKS

1. We will start with an illustration of the geometric meaning of singular value decomposition.
Let us look at a singular value decomposition of a 2 × 2 matrix. Open the file lab16.m,
locate the code cell %% 2x2 matrix, and add the following commands
19
The printout of the file lab14.m can be found in the appendices of this book.
20
The image is available at: https://commons.wikimedia.org/wiki/File:Albert_Einstein_Head.jpg
21
Image from page 446 of ”The standard Hoyle; a complete guide and reliable authority upon all games of
chance or skill now played in the United States, whether of native or foreign introduction” (1909), image appears
on Flickr Commons.

85
 %% 2x2 matrix
clear;
t=linspace(0,2*pi,100);
X=[cos(t);sin(t)];
subplot(2,2,1);
hold on;
plot(X(1,:),X(2,:),’b’);
quiver(0,0,1,0,0,’r’);
quiver(0,0,0,1,0,’g’);
axis equal
title(’Unit circle’)
hold off;

This code will create a 2 × 100 matrix X = [x1 x2 · · · x100 ] whose columns xi are unit
vectors pointing in various directions. The plot will show a blue circle corresponding to the
endpoints of these vectors and two vectors of the standard basis on the plane e1 = (1, 0)T
and e2 = (0, 1)T . We use the function subplot which will create a plot containing four
subplots arranged in two rows and two columns. The plot above will occupy the first “cell”
of this plot. Observe the command quiver which draws a vector with the beginning point
given by the first two arguments ((0,0) in this case) and an ending point given by the
next two arguments ((1,0) or (0,1) in the code above).
Variables: X
2. Now in the M-file, define a variable A holding the matrix
A = [ 2, 1; -1, 1 ];
and compute the singular value decomposition of this matrix using the svd function:
[U,S,V] = svd(A);
Using the workspace window of the Matlab main environment, check out the matrices U,
S, V. Perform the commands
U’*U
V’*V

to ascertain that the matrices U and V are orthogonal. The output should produce 2 × 2
identity matrices.
Variables: A, U, S, V
3. Next, let us observe the geometric meaning of the individual matrices U, Σ, V (U, S, V
in our Matlab code) in the singular value decomposition. To do this, observe the transfor-
mations induced by these matrices on a unit circle and the vectors of the standard basis e1 ,
e2 . Let us start by multiplying the coordinates of the points of the circle and the vectors
e1 , e2 by the matrix V. Execute the following code:
VX=V’*X;
subplot(2,2,2)
hold on;
plot(VX(1,:),VX(2,:),’b’);
quiver(0,0,V(1,1),V(1,2),0,’r’);
quiver(0,0,V(2,1),V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix V^T’)
hold off;

86
 Observe that the matrix VX contains the vectors of the matrix X transformed by the multi-
plication by the matrix VT . Since the matrices V and VT are orthogonal, multiplication
by the matrix VT is equivalent to rotation of a plane, possibly in combination with a
reflection along some straight line. This allows us to conjecture that the image of the unit
circle under this mapping will still be a unit circle, but the vectors of the basis will be
rotated and possibly switched in orientation.
Variables: VX
Q1: Did the multiplication by the transpose of the matrix V resulted in a reflection of
the plane?

4. Now, let us multiply the result from the previous step by the matrix Σ (S in the Matlab
code). Observe that since the matrix Σ is diagonal, then multiplication by this matrix
geometrically means stretching of the plain in two directions. To verify this, execute the
following Matlab code:
SVX = S*VX;
subplot(2,2,3);
hold on;
plot(SVX(1,:),SVX(2,:),’b’);
quiver(0,0,S(1,1)*V(1,1),S(2,2)*V(1,2),0,’r’);
quiver(0,0,S(1,1)*V(2,1),S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix \Sigma V^T’)
hold off;

Observe that, as expected, the unit circle is stretched and becomes an ellipsis. The images
of the standard basis vectors are stretched as well.
Variables: SVX
5. Finally, multiply the results from the last step by the matrix U to obtain:
AX = U*SVX;
subplot(2,2,4)
hold on;
plot(AX(1,:),AX(2,:),’b’);
quiver(0,0,U(1,1)*S(1,1)*V(1,1)+U(1,2)*S(2,2)*V(1,2),U(2,1)*S(1,1)*V(1,1)+...
U(2,2)*S(2,2)*V(1,2),0,’r’);
quiver(0,0,U(1,1)*S(1,1)*V(2,1)+U(1,2)*S(2,2)*V(2,2),U(2,1)*S(1,1)*V(2,1)+...
U(2,2)*S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix U\Sigma V^T=A’)
hold off;

Observe that the result is equivalent to multiplying the initial vector X by the matrix A.
Since the matrix U is orthogonal, then the multiplication by this matrix should result in a
rotation of the plane possibly combined with a reflection. Confirm this by observing the
images of the basis vectors.
Variables AX
Q2: Did multiplication by the matrix U produce a reflection of the plane?

6. If you answered yes to both Q1 and Q2 above, can you modify the matrices U and V in such
a way that no reflections of the plane occur? Produce the modified matrices U1 and V1
and confirm that U1*S*V1’=A. Observe that this shows that singular value decomposition

87
 is not unique.
Variables U1, V1

7. Finally, observe that

 
T σ1 0
[Av1 Av2 ] = AV = USV V = US = [u1 u2 ] = [σ1 u1 σ2 u2 ],
0 σ2

showing that
Av1 = σ1 u1 and Av2 = σ2 v2 .
Check this fact numerically by computing the expression A*V-U*S.
8. Now we will look at image compression using SVD. Add the following commands to your
M-file:
%% Image compression
clear;
ImJPG=imread(’einstein.jpg’);
figure;
imshow(ImJPG);
[m,n]=size(ImJPG);

This code loads an image as a uint8 matrix and displays it on the screen. Each entry in
the matrix corresponds to a pixel on the screen and takes a value somewhere between 0
(black) and 255 (white).
Variables: ImJPG

9. Perform a singular value decomposition of the matrix ImJPG and save the output in matri-
ces UIm, SIm, and VIm. Since ImJPG is integer-valued, you will need to use svd(double(ImJPG)).
Variables: UIm, SIm, VIm

10. Plot the singular values using the code:

figure;
plot(1:min(m,n),diag(SIm));

This shows the singular values (the diagonal entries of the SIm matrix) for the image
matrix ImJPG. Notice that the diagonal entries of SIm are ordered in the decreasing order
σ1 ≥ σ2 ≥ . . . ≥ σmin(m,n) ≥ 0.

11. The idea behind compression of data with singular value decomposition is in the following.
Using the singular value decomposition, the matrix A can be written in the form:

A = σ1 u1 v1T + σ2 u2 v2T + · · · + σr ur vrT

where r is the rank of A and ui and vi are the ith columns of U and V respectively.
Observe that since the singular values are arranged in the decreasing order, the first terms
in this sum provide a larger contribution to the matrix A then the subsequent terms. It
may happen that for some k < r, σk+1 is small compared to σ1 , and correspondingly does
not affect the matrix A too much. We should then expect

A ≈ σ1 u1 v1T + σ2 u2 v2T + · · · + σk uk vkT

88
 to be a good approximation to the matrix A (this is called a truncated SVD). In fact,
there is a theorem in linear algebra which states that SVD truncated to the k terms is
the best approximation to the matrix A among the matrices of the rank at most k in the
sense of Frobenius norm (which is equivalent to the regular Euclidean norm if we consider
the matrix A as an mn-dimensional vector).
This idea can be used for image compression as follows. Instead of storing the whole m × n
matrix A, we can instead store the m × k and n × k matrices
C = [u1 u2 · · · uk ] and D = [σ1 v1 σ2 v2 · · · σk vk ].
If k is much smaller than min(m, n), then storing C and D will take much less space than
storing A. Moreover, if we wish to display the image, we can reconstruct the approximation
of A as A ≈ CDT .
Add the following code to your M-file:
% Approximation of the original matrix with 50, 100, and 150 singular values
for k=50:50:150
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
% compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for %2.0f singular values: %8.3f\n’,k, pct);
end;

This code compresses the image as described above using k = 50, k = 100, and k = 150
singular values and displays the reconstructed image. Compare the reconstructed images
with the original. The code also displays the compression percentage as pct.
Variables: ImJPG comp, pct
12. Observe that the singular value decomposition can also be used to smooth noisy data,
especially if the data contains patterns. Data smoothing is often necessary because all
measurements contain small errors resulting in a “noise”. This noise usually determines
the smallest singular values of the matrix. Dropping these small values, thus, not only
saves the storage space, but also allows to eliminate noise from the data.
Start a new code cell. Load the file checkers.pgm into Matlab and add some noise to the
resulting image matrix using the following code:
%% Noise filtering
clear;
ImJPG=imread(’checkers.pgm’)
[m,n]=size(ImJPG);
ImJPG_Noisy=double(ImJPG)+50*(rand(m,n)-0.5*ones(m,n));
figure;
imshow(ImJPG);
figure;
imshow(uint8(ImJPG_Noisy));

Variables: ImJPG, ImJPG Noisy

13. Compute the SVD of the matrix ImJPG Noisy and save the resulting decomposition ma-
trices as UIm, SIm, and VIm.
Variables: UIm, SIm, VIm

89
14. Compute the approximations of the initial image with k = 10, k = 30, and k = 50
singular values. Display the resulting approximations and compare then to the “noisy”
image. Observe that SVD significantly reduces the noise. Compare the images to the
initial image without noise. Observe also that even though SVD reduces the noise, it also
somewhat blurs the image.

90
Appendices
Printout of the file lab09.m
%%
clear;
Vertices=[1 1 1;...
-1 1 1;...
1 -1 1;...
1 1 -1;...
-1 -1 1;...
-1 1 -1;...
1 -1 -1;...
-1 -1 -1];

Edges=zeros(8,8);
Edges(1,2)=1;
Edges(1,3)=1;
Edges(1,4)=1;
Edges(2,5)=1;
Edges(2,6)=1;
Edges(3,5)=1;
Edges(3,7)=1;
Edges(4,6)=1;
Edges(4,7)=1;
Edges(5,8)=1;
Edges(6,8)=1;
Edges(7,8)=1;

Edges=Edges+Edges’;

theta1=pi/3;
theta2=pi/4;
theta3=pi/6;

rotmat=rotation(theta1,theta2,theta3);

VertRot=Vertices*rotmat;

figure;
axis equal;
hold on;

for j=1:8
for k=j+1:8
if (Edges(j,k)==1)
line([VertRot(j,1),VertRot(k,1)],[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;

hold off;

%% Buckyball

91
[Edges,Vertices] = bucky;
figure;
axis equal;
hold on;

theta1=pi/3;
theta2=pi/4;
theta3=pi/6;

rotmat=rotation(theta1,theta2,theta3);

VertRot=Vertices*rotmat;

for j=1:60
for k=j+1:60
if (Edges(j,k)==1)
line([VertRot(j,1),VertRot(k,1)],[VertRot(j,2),VertRot(k,2)]);
end;
end;
end;

hold off;

figure;
axis equal;
hold on;

for j=1:60
for k=j+1:60
if (Edges(j,k)==1)
line([Vertices(j,1),Vertices(k,1)],[Vertices(j,2),Vertices(k,2)],...
[Vertices(j,3),Vertices(k,3)]);
end;
end;
end;

hold off;

%% 3D fawn
clear;

load(’v.mat’,’v’);
load(’f.mat’,’f’);

[mVert,nVert]=size(v);

[mFace,nFace]=size(f);

figure;
axis equal;
hold on;
for j=1:mFace
line([v(f(j,1),1),v(f(j,2),1)],[v(f(j,1),2),v(f(j,2),2)],[v(f(j,1),3),v(f(j,2),3)]);

92
 line([v(f(j,1),1),v(f(j,3),1)],[v(f(j,1),2),v(f(j,3),2)],[v(f(j,1),3),v(f(j,3),3)]);
line([v(f(j,2),1),v(f(j,3),1)],[v(f(j,2),2),v(f(j,3),2)],[v(f(j,2),3),v(f(j,3),3)]);
end;
hold off;

figure;
axis equal;
hold on;

theta1=pi/3;
theta2=pi/4;
theta3=pi/2;

rotmat=rotation(theta1,theta2,theta3);

VertRot=v*rotmat;

for j=1:mFace
line([VertRot(f(j,1),1),VertRot(f(j,2),1)],[VertRot(f(j,1),2),VertRot(f(j,2),2)]);
line([VertRot(f(j,1),1),VertRot(f(j,3),1)],[VertRot(f(j,1),2),VertRot(f(j,3),2)]);
line([VertRot(f(j,2),1),VertRot(f(j,3),1)],[VertRot(f(j,2),2),VertRot(f(j,3),2)]);
end;

93
Printout of the file lab10.m
% The Chaos Game

%% Sierpinski triangle

%% Sierpinski triangle with a change of ratio

%% Sierpinski triangle with rotation

%% Square

%% Square with different ratio

%% Square with vertex preference (new vertex cannot be the same as previous)
clear;

T=[1/2 0; 0 1/2];

t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=5000;

k1=0;
for j=1:Num
k=randi(3);
if (k>=k1)
k=k+1;
end;
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
k1=k;
end;

plot(x(1,:),x(2,:),’b.’);
hold off;

%% Square with vertex preference (new vertex cannot be opposite of the previous)
clear;

T=[1/2 0; 0 1/2];

t=linspace(0,2*pi,5);
t(5)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

94
plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=5000;

w=[real(v(1));imag(v(1))];
k1=0;

for j=1:Num
k=randi(4)
if (k~=k1+2)&&(k1~=k+2)
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
k1=k;
else
x(:,j+1)=x(:,j);
end;
end;

plot(x(1,:),x(2,:),’b.’);
hold off;

%% Barnsley fern
clear;

T1=[0.85,0.04;-0.04,0.85];
T2=[-0.15,0.28;0.26,0.24];
T3=[0.2,-0.26;0.23,0.22];
T4=[0,0;0,0.16];
Q1=[0;1.64];
Q2=[-0.028;1.05];
Q3=[0;1.6];
Q4=[0,0];
P1=0.85;
P2=0.07;
P3=0.07;
P4=0.01;

Num=15000;
x(:,1)=rand(2,1);
plot(x(1,:),x(2,:),’b.’)
axis equal;
hold on;
for j=1:Num
r=rand;
if r<=P1
x(:,j+1)=T1*x(:,j)+Q1;
elseif r<=P1+P2
x(:,j+1)=T2*x(:,j)+Q2;
elseif r<=P1+P2+P3
x(:,j+1)=T3*x(:,j)+Q3;
else
x(:,j+1)=T4*x(:,j);

95
 end
end

plot(x(1,:),x(2,:),’b.’);
hold off;

%% Hexagon
clear;

T=[1/3 0; 0 1/3];

t=linspace(0,2*pi,7);
t(7)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=10000;

for j=1:Num
k=randi(6);
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

%% Pentagon with a skipped vertex

clear;
T=[1/2.5 0; 0 1/2.5];

t=linspace(0,2*pi,6);
t(6)=[];
v=[cos(t); sin(t)];

x(:,1)=[rand-0.5; rand-0.5];

plot(v(1,:),v(2,:),’k*’,x(1,1),x(2,1),’b.’);
axis equal;
hold on;

Num=10000;

k1=0;

for j=1:Num
k=randi(5);
if k~=k1
x(:,j+1)=T*(x(:,j)-v(:,k))+v(:,k);

96
 k1=k;
else
x(:,j+1)=x(:,j);
end;
end;
plot(x(1,:),x(2,:),’b.’);
hold off;

97
Printout of the file lab11.m
% Simple face recognition algorithm
%% Input database files into the Matlab
clear;
Database_Size=30;

%Reading images from the database. The image files should be located in the
%subfolder "database"
for j=1:Database_Size
image_read=imread([’database\person’ num2str(j) ’.pgm’]);
[m,n]=size(image_read);
P(:,j)=reshape(image_read,m*n,1);
end;

%% Computing and displaying the mean face

%% Subtract the mean face

%% Compute the covariance matrix of the set and its eigenvalues and eigenvectors

%% Display the set of eigenvaces

for j=2:Database_Size;
if j==2
EigenFaces=reshape(EigenVectors(:,j)+mean_face,m,n);
else
EigenFaces=[EigenFaces reshape(EigenVectors(:,j)+mean_face,m,n)];
end;
end

EigenFaces=uint8(EigenFaces);

figure;
imshow(EigenFaces);

%% Orthogonality and symmetry

%% Recognition of an altered image (sunglasses)

image_read=imread([’database\person30altered1.pgm’]);
U=reshape(image_read,m*n,1);

NormsEigenVectors=diag(Products);
W=(EigenVectors’*(double(U)-mean_face));
W=W./NormsEigenVectors;

U_approx=EigenVectors*W+mean_face;

image_approx=uint8(reshape(U_approx,m,n));
figure;
imshow([image_read,image_approx])

%% Recognition of an altered image (lower part of the face)

%% Recognition of a person not in the database

98
Printout of the file lab12.m
% Google PageRank algorithm on the example of random network

%% Load the network data

clear;

%% Display a small amount of network

NumNetwork=500;
AdjMatrixSmall=AdjMatrix(1:NumNetwork,1:NumNetwork);

for j=1:NumNetwork
coordinates(j,1)=NumNetwork*rand;
coordinates(j,2)=NumNetwork*rand;
end;

gplot(AdjMatrixSmall,coordinates,’k-*’);

%% Check the amount of links originating from each webpage

NumLinks=sum(AdjMatrixSmall,2);

%% Create a matrix of probabilities (Google matrix)

% Element (i,j) of the matrix shows the probability of moving from i-th
% page of the network to jth page. It is assumed that the user can follow
% any link on the page with a total probability of 85% (all hyperlinks are
% equal), and jump (teleport) to any other page in the network with a total
% probability of 15% (again, all pages are equal).

alpha=0.15;
GoogleMatrix=zeros(NumNetwork,NumNetwork);
for i=1:NumNetwork
if NumLinks(i)~=0
GoogleMatrix(i,:)=AdjMatrixSmall(i,:)./NumLinks(i);
else
GoogleMatrix(i,:)=1./NumNetwork;
end;
end;

GoogleMatrix=(1-alpha)*GoogleMatrix+alpha*ones(NumNetwork,NumNetwork)./NumNetwork;

%% Check that all the rows in the GoogleMatrix matrix sum to 1

SumGoogleMatrix=sum(GoogleMatrix,2);

%% Finding an eigenvector corresponding to 1 (why is there sucj an eigenvector)?

w0=ones(1,NumNetwork)./sqrt(NumNetwork);

w1=w0*GoogleMatrix;
w2=w1*GoogleMatrix;
w3=w2*GoogleMatrix;

w100=w0*(GoogleMatrix)^100;
w90=w0*(GoogleMatrix)^90;

99
% Check the difference between v30 and v20. Observe that it is pretty
% small
deltaw=w100-w90;

%% Compute the eigenvalues and the right eigenvectors

[VectRight,LamRight]=eig(GoogleMatrix);
% Explain the result
LamRight=diag(LamRight);

%% Compute the eigenvalues and the left eigenvectors

[VectLeft,LamLeft]=eig(GoogleMatrix’);
LamLeft=diag(LamLeft);

%% Separate the eigenvector corresponding to the eigenvalue 1 and scale it

u1=VectLeft(:,1);

u1=abs(u1)/norm(u1);

%% Select the maximum element and the corresponding element.

%Which page is the most important in the network?

[MaxRank,PageMaxRank]=max(u1);

%% Check if it’s the most popular (most linked to page):

MostLinks=sum(AdjMatrixSmall);
[MaxLinks,PageMaxLinks]=max(MostLinks);

100
Printout of the file lab13.m
% Facebook and graph partitioning

%% Simple graph

%% Load the data

clear;
load(’social.mat’,’Social’);
spy(Social);

%% Laplacian

%% Spectral decomposition

%% Clusteer in two groups

%% Cluster in 4 groups

Diagonal(indsmall)=large+1000;
[small,indsmall]=min(Diagonal);

V3=V(:,indsmall);

pp=[];
pn=[];
np=[];
nn=[];

for j=1:m
if (V2(j)>0)
if (V3(j)>0)
pp=[pp,j];
else
pn=[pn,j];
end;
else
if (V3(j)>0)
np=[np,j];
else
nn=[nn,j];
end;
end;
end;

order=[pp,pn,np,nn];

iden=eye(m,m);

for j=1:351;
for k=1:351;
P(j,k)=iden(order(j),k);
end;
end;

101
SocialOrdered=P*Social*P’

figure;
spy(SocialOrdered)

%% Second order of Fiedler

SocialPos=Social(pos,pos);
SocialNeg=Social(neg,neg);

rowsumpos=sum(SocialPos,2);

DiagSocialPos=diag(rowsumpos);

LaplaceSocialPos=DiagSocialPos-SocialPos;

[VPos,DPos]=eig(LaplaceSocialPos);

V2Pos=VPos(:,2);

[mpos,npos]=size(V2Pos);

posp=[];
posn=[];

for j=1:mpos
if (V2Pos(j)>0)
posp=[posp,pos(j)];
else
posn=[posn,pos(j)];
end;
end;

rowsumneg=sum(SocialNeg,2);

DiagSocialNeg=diag(rowsumneg);

LaplaceSocialNeg=DiagSocialNeg-SocialNeg;

[VNeg,DNeg]=eig(LaplaceSocialNeg);

V2Neg=VNeg(:,2);

[mneg,nneg]=size(V2Neg);

negp=[];
negn=[];

for j=1:mneg
if (V2Neg(j)>0)
negp=[negp,neg(j)];
else
negn=[negn,neg(j)];

102
 end;
end;

ordergen=[posp,posn,negp,negn];

iden=eye(m,m);

for j=1:351;
for k=1:351;
P(j,k)=iden(ordergen(j),k);
end;
end;

SocialOrderedGen=P*Social*P’
figure;
spy(SocialOrderedGen)

103
Printout of the file lab14.m
%% 2x2 matrix

clear;
t=linspace(0,2*pi,100);
X=[cos(t);sin(t)];
subplot(2,2,1);
hold on;
plot(X(1,:),X(2,:),’b’);
quiver(0,0,1,0,0,’r’);
quiver(0,0,0,1,0,’g’);
axis equal
title(’Unit circle’)
hold off;

A = [ 2, 1; -1, 1 ];
[U,S,V] = svd(A);
U’*U
V’*V

VX=V’*X;
subplot(2,2,2)
hold on;
plot(VX(1,:),VX(2,:),’b’);
quiver(0,0,V(1,1),V(1,2),0,’r’);
quiver(0,0,V(2,1),V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix V^T’)
hold off;

SVX = S*VX;
subplot(2,2,3);
hold on;
plot(SVX(1,:),SVX(2,:),’b’);
quiver(0,0,S(1,1)*V(1,1),S(2,2)*V(1,2),0,’r’);
quiver(0,0,S(1,1)*V(2,1),S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix \Sigma V^T’)
hold off;

AX = U*SVX;
subplot(2,2,4)
hold on;
plot(AX(1,:),AX(2,:),’b’);
quiver(0,0,U(1,1)*S(1,1)*V(1,1)+U(1,2)*S(2,2)*V(1,2),U(2,1)*S(1,1)*V(1,1)+...
U(2,2)*S(2,2)*V(1,2),0,’r’);
quiver(0,0,U(1,1)*S(1,1)*V(2,1)+U(1,2)*S(2,2)*V(2,2),U(2,1)*S(1,1)*V(2,1)+...
U(2,2)*S(2,2)*V(2,2),0,’g’);
axis equal
title(’Multiplied by matrix U\Sigma V^T=A’)
hold off;

%% Modified SVD

104
U1(:,1)=U(:,1);
U1(:,2)=-U(:,2);
V1(:,1)=V(:,1);
V1(:,2)=-V(:,2);

U1*S*V1’-A

%% Check
A*V-U*S

%% Image compression

% Creates a two-dimensional array with the dimensions equal to the dimensions of

% the image
clear;
ImJPG=imread(’einstein.jpg’);
figure;
imshow(ImJPG);

[m,n]=size(ImJPG);

% Compute an SVD

[UIm,SIm,VIm]=svd(double(ImJPG));

% plot the singular values

figure;
plot(1:min(m,n),diag(SIm));

%% Create approximations to the image

% With 50, 100, and 150 singular values

for k=50:50:150
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
% compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for %2.0f singular values: %8.3f\n’,k, pct);
end;

%% Noise filtering
clear;
ImJPG=imread(’checkers.pgm’)
[m,n]=size(ImJPG);

% Add some noise to the image

ImJPG_Noisy=double(ImJPG)+50*(rand(m,n)-0.5*ones(m,n));
figure;
imshow(ImJPG);

figure;
imshow(uint8(ImJPG_Noisy));

105
[UIm,SIm,VIm]=svd(ImJPG_Noisy);

figure;
plot(1:min(m,n),diag(SIm),’ko’);

for k=10:20:50
ImJPG_comp=uint8(UIm(:,1:k)*SIm(1:k,1:k)*(VIm(:,1:k))’);
figure;
imshow(ImJPG_comp)
% compression percentage
pct = 1 - (numel(UIm(:,1:k))+numel(VIm(:,1:k)*SIm(1:k,1:k)))/numel(ImJPG);
fprintf(’Compression percentage for %2.0f singular values: %8.3f\n’,k, pct);
end;

106
References
[1] S. Attaway. Matlab: A practical Introduction to Programming and Problem solving.
Butterworth-Heinemann, 3rd edition, 2013.

[2] K. Bryan, T. Leise. The $25,000,000,000 Eigenvector: The Linear Algebra behind Google.
SIAM Rev., 48(3), 569 - 581, 2006.

[3] T. Chartier. When Life is Linear: From Computer Graphics to Bracketology. Mathematical
Association of America, 2015.

[4] W.N. Colley. Colley’s bias free college football ranking method: The Colley matrix ex-
plained. www.Colleyrankings.com, 2002.

[5] L. Elden. Numerical linear algebra in data mining. Acta Numerica, 15, 327-384, 2006.

[6] C. Hennig, M. Meila, F. Murthagh and R. Rocci (eds.). Handbook of Cluster Analysis.
CRC Press, 2015.

[7] J. Keener. The Perron–Frobenius Theorem and the Ranking of Football Teams. SIAM Rev.,
35(1), 80-93, 1993.

[8] A.N. Langville, C.D. Meyer. Google’s PageRank and Beyond: The Science of Search En-
gine Rankings. Princeton University Press, 2006.

[9] J. Leskovec, R. Sosic. SNAP: A General-Purpose Network Analysis and Graph-Mining

Library,. ACM Transactions on Intelligent Systems and Technology (TIST), 8(1), 2016.

[10] F.M. Harper, J.A. Konstan, The MovieLens Datasets: History and Context. ACM Trans-
actions on Interactive Intelligent Systems (TiiS), 5(4), 2015.

107