HOMEWORK ASSIGNMENTS 2DBI00

MICHIEL HOCHSTENBACH, TU EINDHOVEN, 2024

This homework forms, together with the slides, the heart of the course. The group
assignments and the online homework are also important, but these homework
assignments are even more important. Unless explicitly stated otherwise, do the exercises
by hand, and not using computer programs. The main reason for this is that you have to be
able to do it by hand on the exam. For many questions you can check your answer using your
laptop (and Python, Julia, Matlab, R, C++, Wolfram Alpha, . . . ). Answers to these questions
will usually be made available the evening after the instruction class.

You do not have to hand in your homework. You are encouraged to work together with
classmates, but make sure that you understand the material yourself.

Class 1: Matrices and vectors

(1) Form groups of 4–6, and register your group on Canvas before the deadline.
 
1 2 4 3
(2) Compute AT , A + B, A B, AB and BA for A = 3 4
and B = 2 1
.
 
1 2 1
(3) (a) Compute Ax, xTA and xTAx for A = 3 4
and x = 2
.

(b) What can you conclude about the sizes (= dimensions) of these 3 ?

1
(4) (a) Compute xT x and x xT for x = 2
.
(b) Which of these 2 have you already seen in the Calculus course? What did it mean?
(5) (a) Show that if A and B are 2 ⇥ 2 upper triangular matrices, then AB is also upper
triangular.
(b) Does the opposite also hold: if AB is upper triangular, are A and B upper triangular?
(6) (a) If A and B are 2 ⇥ 2 diagonal matrices, show that AB is also a diagonal matrix.
(b) Is this also true when A and B are n ⇥ n diagonal matrices?
(7) When A is an n ⇥ n matrix, show that B = A + AT is a symmetric matrix.
(8) (a) Find 2 ⇥ 2-matrices A, B 6= O such that AB = O. (Try matrices with many 0’s and
some 1’s.) Is this also possible when A and B are real numbers (1 ⇥ 1 matrices)?
(b) Find a 2 ⇥ 2-matrix A with A 6= O and A2 = O.
Is this also possible when A is a real number (so a 1 ⇥ 1 matrix)?
(c) Find a 2 ⇥ 2-matrix A with A 6= I and A 6= I such that A2 = I.
Is this also possible when A is a real number (so a 1 ⇥ 1 matrix)?
(9) When A is an n ⇥ n matrix with A 6= O, is it possible to find an ↵ 6= 0 such that ↵A = O ?
(10) When A is a symmetric n ⇥ n matrix, show that B = ↵A is also a symmetric matrix.
(11) When A is an n ⇥ n matrix, show that ATA is a symmetric matrix.
NB: this exercise is important! It teaches you how to manipulate with symbols
in a linear algebra way. If you write this out in matrix elements it becomes
very messy, but there is a very elegant proof without this.
(12) When A and B are symmetric 2 ⇥ 2 matrices, is AB also a symmetric matrix ?
Prove, or give a counterexample.
(13) Do exercises (2) and (3) using your computer, for instance making use of Python, Julia,
 Matlab, R, C++, or Wolfram Alpha.
(14) Optional: past exam questions on this class

Class 2: Vector and matrix norms

(1) If you have not done so already, SEE QUESTION (1) OF CLASS 1 !

1
(2) Let x = 1
. Normalize x such that: (a) kxk1 = 1; (b) kxk2 = 1; (c) kxk1 = 1.

1
(3) Let x = 2
. Determine kxk1 , kxk2 , and kxk1 . Which is the largest and the smallest?
Will this be the case for any vector? (See also exercise (9) on “equivalent” norms.)

1 2
(4) Let A = 3 4
. Determine kAk1 , kAk1 , and kAkF . Which norm is the largest?
Which is the smallest? Will this be the case for any matrix? Why?
Background: Later in this course we will learn how to compute the most important kAk2 .
This quantity is related to the important “SVD”.
(5) For A and x mentioned in the previous 2 exercises, compute Ax and check that indeed
kAxk1  kAk1 · kxk1 and kAxk1  kAk1 · kxk1 .
(6) Let x 2 R2 . What can you conclude if it is given that kxk1 = kxk1 ?
(7) Explain why kAk1 = kAT k1 for every matrix A 2 Rm⇥n .
Does it also hold that kAkF = kAT kF ?
(8) Let x, y 2 R2 and ↵ 2 R. For each of the properties below, determine whether they are
true or false. When they are true, give a simple proof to show why. When they are false,
give a counterexample (an example in which the statement is incorrect), and give a correct
statement instead.
(a) k↵xk1 = ↵ kxk1 (b) k xk2 = kxk2
(c) kx + yk1 = kxk1 + kyk1 (d) kAxk1 = Ax

(9) This is a bit challenging but important exercise. It turns out that all norms in Rn are
equivalent, which means that one norm can be bounded from below and above by another.
Let x 2 R2 . For each part below, find the maximal > 0 (lower bound) and the minimal
> 0 (upper bound) number so that the inequalities are valid for all x ( and may be
di↵erent for each part).
(a) kxk1  kxk1  kxk1 (b) kxk1  kxk2  kxk1
(c) kxk1  kxk2  kxk1

(10) What do you think will happen in the previous question if the vector is 2 Rn (instead of
2 R2 )? You don’t have to prove this. Hint: the change is quite “logical”.
(11) Do the following quantities define a vector norm for vectors [x, y]T 2 R2 ?
(a) 2x + y (b) |2x + y| (c) 2 |x| + |y| (d) x2 + |y| (e) max(2 |x|, |y|)
(12) Do exercises (3) and (4) using your computer.
(13) This is a good time to do the group assignment on movie tastes (recommender systems)
together with your group!
(14) Optional: past exam questions on this class
 Class 3: Google PageRank and Markov chains

(1) We have learned that the solution to Ax = x is an eigenvector corresponding to an eigenvalue

= 1. In class 4 we will learn about linear systems. A linear system is of the form Cx = b,
where C and b are given and we have to solve for x.
Explain why the PageRank vector that satisfies Ax = x can also be viewed as a solution to
a certain linear system Cx = b. What are C and b in this case?
Background: we will treat the important topic of linear systems in more detail in the next
class. You can use the answer to this question for the next 2 questions.
(2) Consider the smallest possible internet: 2 sites, with a link from site 1 to site 2. Determine
the connectivity matrix G, then G b by making sure all columns have 1-norm equal to 1.
b (with equal probability
Since one column of G has only zeros, fill this column with 12 ’s in G
to site 1 and 2). Subsequently, let A = 0.85 · G b + 0.15 · · ones(2), where ones(2) is the
1
2
2 ⇥ 2 matrix with all elements equal to 1. Compute by hand and a simple calculator the
eigenvector corresponding to eigenvalue = 1. (If you don’t have a calculator, you may use
the computer, but only use elementary operations; don’t use an eigenvalue routine.) What
are the PageRanks of the 2 sites? Is this as expected?
(3) Now consider the same internet: 2 sites, but now with a link from site 1 to site 2 and vice
versa. Determine G, G,b and A similar to the previous question. Compute by hand the
eigenvector corresponding to eigenvalue = 1. What are the PageRanks of the 2 sites?
Is this as expected?
(4) Why does it make sense for Google PageRank that the diagonal of G contains only zeros?
2 3
0 1 0 1
61 0 1 07
(5) For the connectivity matrix G = 6
40 1 0 05
7, compute the PageRank on the computer.
0 0 1 0
Is the result as expected? 2 3
0 0 0 0
61 0 0 07
(6) Consider the connectivity matrix G = 6
41 1 0 05
7. What is special about site 4?
1 1 1 0

What do you expect that the PageRank will look like?

Now compute the PageRank on the computer. Is the result as expected?
(7) In question (5), how many di↵erent paths are there from site 3 to site 2 in exactly 3 steps?
In question (6), how many di↵erent paths are there from site 1 to site 4 in  3 steps?
Use both powers of G (on the computer) and counting by hand using a picture of the graph.
b is a column-stochastic matrix, and 1 · ones(n) is a column-stochastic matrix as well,
(8) If G n
b + 0.15 · 1 · ones(n) is also a column-stochastic matrix.
show that A = 0.85 · G n
(9) Explain that if a connectivity matrix G is symmetric, one may also view the graph as
undirected, which means that the edges “have no arrows”.
(10) Player X and Y start both with e2. Each turn X gets e1 from Y with probability p, or
instead Y gets e1 from X with probability q = 1 p. The game ends when X or Y has e4.
(a) Sketch the Markov chain and give the column-stochastic matrix A.
(b) Compute the eigenvalues using your computer for any 0 < p < 1 (e.g. p = 0.5). Does
this matrix satisfy the Perron–Frobenius theorem? Can you explain this?
(c) Suppose p = 0.5. What are the probabilities after 10 turns that: X has won; Y has
won; the game is undecided? Do the same for p = 0.6.
(d) Suppose p = 0.51. What are the probabilities that X will win; Y will win the game?
(11) This is a good time to do the group assignment on the Google PageRank type ranking
together with your group!
 Class 4: Linear systems Do all exercises by hand, except for (7) and (8).

(1) Give the reduced row echelon form (rref) of the matrices:
2 3 2 3
 2 2 6 2 8 8 12 30
4 16 44
(a) 1 0 1
(b) 4 4 6 10 5 (c) 4 5 22 8 62 55 5
4 8 8 3 16 9 76 11

(2) Determine by means of an augmented matrix the solution to the systems of equations:
2 3
2 32 3 2 3 2 1 3 3 3
x 3y = 8 3 3 3 x 6 6 3 3 3 0 18 7
(a) (b) 4 1 1 754 y 5 =4 45 (c) 6
4 3 2 2 2
7
13 5
3x 8y = 21 3 2 1 z 3
3 0 3 3 24

(3) Consider the following systems of linear equations with parameter a in the variables
x1 , x2 , x3 , and x4 : 2x1 + x2 + 5x3 + 8x4 = 10
x2 + 3x3 + 4x4 = 4
x1 + x2 + 4x3 + 6x4 = a
(a) For which value(s) of a does this system have a solution?
(b) Give a parameter representation of all solutions if a = 7.
(4) Consider the following system of linear equations with parameter c and variables x1 , x2 ,
and x3 : x1 + 2x2 + 3x3 = 4
2x1 + cx2 + 3x3 = 5
3x1 + 6x2 + cx3 = 9
(a) Check if the system has a solution for c = 9.
(b) Give all solutions of the system for c = 4.
(c) For which values of c does the system have exactly 1 solution?
(5) This question is important for your understanding.
Let A 2 R3⇥3 and x, y, b 2 R3 . Suppose that Ax = b and also Ay = b.
Does it follow that x = y ?
If so, why? If not, under which condition does it hold?
(6) Fruit is important for hard-working students; let’s buy some apples (a), bananas (b), cherries
(c) and dates (d) (NL: dadels). Total cost of one of each type: e9; a + 2b + 3c + 4d = e22.
4a + 3b + 2c + 2d = e25. a + c are e3 more expensive than b + d.
What does each item cost?
(7) Compute 1(c) using your computer, e.g., by using a command of the form
rref([1 2; 3 4]) in Matlab or rref {{1,2},{3,4}} in Wolfram Alpha.
(In Python this is also possible, but less standard. It may require some extra functionality of the form:
from sympy import Matrix, A = Matrix([[1., 2., 3.], [2., 4., 5.]]), A.rref()
The first output is the rref-form, while the second is the columns that contain pivots.)

(8) Compute 2(a) and 2(b) using your computer. Do you get the right solutions?
(The (b) part is tricky since it has 1 many solutions. Depending on the program you use,
it may or may not work. Wolfram Alpha seems to do fine, how about Python or Matlab?)
(9) Can you think of examples where one needs to solve linear systems in your field of study?
(10) Solve the parametrized linear systems on the final slides of Class 4.
(11) Optional: past exam questions on this class
Class 5: Linear algebra on your computer

(1) Let A, B 2 Rn⇥n , x, y 2 Rn , ↵ 2 R. In this question, assume that A and B are full (also
called dense; not sparse). The number of flops to compute xT y is 2n 1; ignoring the less
important term we can say: ⇡ 2n. In the same way, give the numbers of flops for:
(a) Ax + y (c) ↵Ax (e) yTAx
(b) (A + B) x (d) ↵ABx (f ) ↵ yT x
Important! Parts (b)–(f ) can be computed in several ways. Show that you have chosen
the fastest way, by comparing the numbers of flops.
(2) Repeat (a)–(e) of the previous exercise, but now for the practically important case that A
and B are tridiagonal (so bandwidth ` = 1).
(3) Let x, y, z be machine numbers (which means that we consider them as exact, and do not
look at rounding errors from storing a real number x on the computer). Let z 6= 0.
What is the maximal relative error in xy
z ? Give a rounding error analysis with ’s.
(4) Let x and y be machine numbers. We are going to see that on a computer, x2 y 2 may be
di↵erent from (x + y)(x y) !
(a) Carry out a rounding error analysis with ’s, and show that the rounding errors in
x2 y 2 may be larger than (x + y)(x y) when x and y are large and x ⇡ ±y.
(b) Now on the computer try a command of the form (and compare a and b)
x = pi*1e14; y = x-0.1; a = x^2-y^2; b = (x+y)*(x-y); (a-b)/b
^ means “to the power”; in Python, this should be replaced by **.
(5) We now look at the famous math identity sin2 (x) + cos2 (x) = 1, which is not necessarily
true on a computer!
(a) Using an analysis with ’s, give the maximal rounding error.
(b) Now try a command such as x = 0.17; sin(x)^2 + cos(x)^2 - 1 on the computer.
(6) Cancellation: suppose x = 0.001 has a relative error of at most 5%, and y = 0.00099 has a
relative error of at most 1%, what is the maximal relative error in x y ?
(7) Consider Table 1. Explain the following answers given by a computer, using standard IEEE
double precision arithmetic. Here, eps = 2 52 ⇡ 2.2 · 10 16 .
== tests whether the left-hand side and right-hand side are equal.
Table 1 Table 2
Command Answer Command Answer
1+eps == 1 false 2+eps == 2 ??
1+eps/2 == 1 true sqrt(2)+eps == sqrt(2) ??
2^-1000 9.3326e-302 1+eps-1 ??
2^-1100 0 1e16 +1 -1e16 ??
2^1000 1.0715e+301 1/2 == 0.5 ??
2^1100 Inf 0.6/0.2 == 3 ??
(8) Now consider Table 2. First try to predict what answers your computer will give.
Then try them on the computer to see if you are right. (For eps ⇡ 2.2 · 10 16 : see slides.)
(9) Optional: past exam questions on this class
 Class 6: Inverse matrices

(1) Determine whether the following matrices are nonsingular. If so, determine the inverse
2 3
matrix.

2 3 2 3 1
6 5 4 6 5 4 6 7
1 2 2
(a) 3 4
(b) 4 2 1 25 (c) 4 2 1 25 (d) 6
4 3
7
5
4 4 2 4 4 0
2 3 2 3 4
1 1
(e) 4 15 (f ) 42 1 5 (g) 2 I1000
1 3 4 1

(2) Explain what you see when you look at the answers to parts 1(d), 1(e), 1(f ).
(3) In view of question (1), what is the inverse of the n ⇥ n matrices
2 3
a1
..
(a) 4 . 5 (all ai 6= 0) (b) ↵ In (↵ 6= 0) (c) ⇧ (permutation matrix) ?
an

(4) Let A be a nonsingular n ⇥ n matrix, and b 2 Rn an n-vector, and consider the linear
system Ax = b. One can solve the system and compute x = A 1 b in two ways.
Explain which two, and which method is the most efficient in which situation.
2 3 2 3
2 1 0 4
(5) Now carry out both methods of question (4) on A = 41 2 15 and b = 425
0 1 2 8

(6) If A is a square nonsingular matrix, then what would (A 1) 1 be ?


1 2
Check your hypothesis on A = 3 4

(7) Let A be nonsingular. Find the inverse of A2 , by which you also show that A2 is also
nonsingular.
(8) Let A be symmetric and nonsingular. Is the inverse A 1 also symmetric?
Show, or give a counterexample.
(9) Which is easier: computing B = A 1, or checking if a given B is the inverse of A:
(a) by hand? (b) by computer?
(10) This question is very important for your understanding of the material.
Let A 2 Rn⇥n . Explain why the following 5 statements are equivalent:
A is nonsingular () A has n pivots () rref(A) = In
() Ax = b has a unique solution x = A 1 b for every b 2 Rn
() Ax = 0 has only x = 0 as solution.
(Background: in the next classes we will come back with more equivalent statements.)
(11) Now answer (1)(c) on your computer.

a b
(12) Challenge: By row reduction on the general 2 ⇥ 2 matrix A = c d
, deduce the formula

1 1 d b
for the inverse A = ad bc c a
.
Tip: Start with [A | I] and try to arrive at [I | A 1 ]. While solving this question, do not
worry whether quantities in the denominator are 0, check this only at the end. Check your
answer on 1(a).
This formula may be useful in several situations, so it might be an idea learning it by heart.
(13) Optional: past exam questions on this class
 Class 7: Determinants

(1) What is the determinant of the 3 ⇥ 3 diagonal matrix diag(↵, , ) ?

And of the general n ⇥ n diagonal matrix diag(a1 , . . . , an ) ?
2 3
a b c
(2) What is the determinant of the 3 ⇥ 3 upper triangular matrix 40 d e5 ?
0 0 f
And of the general n ⇥ n upper triangular matrix with (a1 , . . . , an ) on the diagonal ?
 
1 3
(3) What is the area of the parallelogram spanned by 3
and 1
? (So the other points are
 
0 4
the origin 0
and 4
.) Can you determine this easily by another method?
2 3 2 3 2 3
1 1 0
(4) What is the volume of the parallelepiped spanned by 4 2 5, 4 2 5, and 405 ?
2 3 4

a b e
(5) When does the 2 ⇥ 2 matrix c d f
have a unique solution?
Give a constraint in terms of the coefficients. What is notable about the condition?

1 3
(6) When is the 2 ⇥ 2 matrix 2 ↵
singular? Why is this natural?
2 3
 ↵+3 3 1
↵+2 1
(7) For which values of ↵ is: (a) ↵+2 ↵ 2
singular? (b) 4 8 ↵ 6 1 5 singular?
4 0 ↵ 3

(8) Let A, B, and C be n ⇥ n matrices, and I the n ⇥ n identity matrix. For each of the
properties below, determine whether they are true or false. When they are true, give a
simple proof to show why. When they are false, give a counterexample (an example in
which the statement is incorrect), and give a correct statement instead.
(a) det(AB) = det(BA) (d) det(ABC) = det(A) det(B) det(C)
(b) det( I) = 1 (e) det(A2 ) > 0
1) det(A)
(c) det(A) = 0 =) A = O (f ) det(AB = det(B)

(9) Show that for 1 ⇥ 1 matrices we have det(A + B) = det(A) + det(B), but show by giving a
2 ⇥ 2 example that this generally does not hold for matrices of larger size.
2 3
1 2
6 3 4 7
6 7
(10) Compute by hand the determinant of 6 5 7. What does the answer mean?
4 4 3 5
2 1
2 3
F F R R
6F R R7
6 7
(11) Compute by computer the determinant of 6F F R R 7 where F = 1 and R = 2.
4F R R 5
F R R
(Can you recognize the initials of a popular soccer club in The Netherlands?)
Can you explain the result? (Hint: the answer is the volume of a 5-dimensional object.
What do you see when you look at the points defined by the 5 columns ?)
(12) Important for your understanding:
Explain briefly to yourself the reason why we can efficiently check on det(A) = 0 to see if a
matrix is singular. Use the notions of “elementary matrices”, “pivots”, and “rref”.
(13) Optional: past exam questions on this class
 Class 8: Eigenvalues and eigenvectors
2 3
 1 3 7
2 1
(1) What are the eigenvalues and eigenvectors of: (a) 3
(b) 4 2 15
3

(2) What are the eigenvalues of the general n ⇥ n upper triangular matrix with (a1 , . . . , an ) on
the diagonal? Of the eigenvectors, 1 is very easy; which one? (See (1)(b))
2 3
 4 3 1
4 2
(3) What are the eigenvalues and eigenvectors of: (a) 2 4
(b) 4 1 0 55
1 3 2
Hint: for (b) one of the eigenvalues is = 3.
" 1
p 1
#
3
(4) Show that the matrix 2
1 1
2
p has no real eigenvalues.
2 2
3
Background: We will see later that this is a rotation matrix over ⇡6 (= 30 degrees).
A rotation leaves no vector in place, so it is natural that there is no real eigenvalue/vector.
(5) It can be shown that the eigenvalues of a symmetric matrix are real, that

is, cannot be
d e
complex. Show this for a 2 ⇥ 2 matrix: prove that the eigenvalues of A = e f must be
real. Hint: look at det(A I) = 0 and use the quadratic formula (“abc-formule”).
(6) Can a 3 ⇥ 3 (real) matrix have: (a) 3 real eigenvalues? (b) 2 real eigenvalues and 1
complex? (c) 1 real eigenvalue and 2 complex? (d) 3 complex eigenvalues?
NB: by “complex”, we mean “non-real”. Hint: consider the characteristic polynomial
p( ) = det(A I), which is of degree 3, and use your Calculus expertise!
(7) Suppose ( , x) is an eigenpair of a A. Show the following:
(a) ( ↵, x) is an eigenpair of A ↵I (c) ( 2 , x) is an eigenpair of A2
(b) (↵ , x) is an eigenpair of ↵A (d) ( 1 , x) is an eigenpair of A 1

(when A is nonsingular)

2 1
(8) Illustrate all parts of the previous question by considering A = 3
, ↵ = 3, = 2, and
the corresponding eigenvector x.
(9) Show: the eigenvalues of AT are the same as the eigenvalues of A.
Hint: Look at det(A I) and det(AT I).
(10) Show that a row-stochastic matrix (all row sums are 1) has an eigenvalue = 1.
Hint: find an “easy” vector x such that Ax = x. (By the previous question, this is also
true for a column-stochastic matrix; this gives a link with Google PageRank.)
(11) Let A, B 2 Rn⇥n with B nonsingular. Show that the eigenvalues of AB are exactly those
of BA. So although AB is usually 6= BA, their eigenvalues are the same.
Hint: Consider det(B) det(AB I) det(B) 1 .
(12) Let A, B, C be n ⇥ n matrices with B nonsingular. Let C = BAB 1 , i.e., C is similar to A.
(a) Show that the eigenvalues of C are the same as the eigenvalues of A.
(b) Show that x is an eigenvector of A corresponding to eigenvalue if and only if Bx is
an eigenvector of
2
C. 3 2 3
2 2 3 2 1 1
(c) Show that A = 4 2 1 65 and C = 4 4 1 45 are similar.
3 6 1 9 9 1
Hint: Try to find a suitable diagonal matrix B.
(d) Now conclude that the eigenvalues of C are real, without actually computing them.
(13) Compute (3)(b) on your computer.
(14) Optional: past exam questions on this class
 Class 9: Clustering via the Fiedler vector; eigenvalue location

(1) Compute using the computer 2 clusters according to the Fiedler vector
of this graph. Do you think the result is logical?
(2) Explain why every Laplacian matrix has an eigenvalue = 0.
What is the corresponding eigenvector?
(3) Explain why the fact that a Laplacian matrix is of the form B T B (see slides) implies that
all eigenvalues are 0.
(4) Explain why Gershgorin’s Theorem, applied to a Laplacian matrix, implies that all eigen-
values are 0.
(5) What can you say using Gershgorin’s Theorem about the maximum eigenvalue of a Lapla-
cian matrix? And therefore about all eigenvalues?
(6) Compute by hand the Fiedler clustering of the undirected graph with 2 nodes and 1 edge.
Do you think the result is logical?
(7) Compute by hand the Fiedler clustering of the undirected graph with 3 nodes and 2 edges:
one from node 1 to node 2, and one from node 2 to node 3. Do you think the result is
logical?
(8) Compute by computer the result for the previous question, but now the edge connecting
nodes 1 and 2 has double weight.
2 3
3 1
(9) Consider 4 1 4 1 5. Why are the eigenvalues of this matrix real? Using Gershgorin,
1 5
what is the smallest set in which the eigenvalues are guaranteed to be located?
2 3
3 0.1
(10) The same questions as the last exercise, but now for 4 0.1 4 0.1 5.
0.1 5
(11) Using kAk1 , kAk2 , and kAk1 , what info do we get about the eigenvalues of A in (9)?

4 2
(12) Show that A = 2 4
is positive definite in 2 di↵erent ways:
 
x1 0
(a) by Calculus techniques: show that xTAx > 0 for x = x2
6= 0
(b) by computing the eigenvalues.
(13) Show in an easy way that the column
S version P
of Gershgorin’s theorem is also true:
all eigenvalues are contained in i { |x aii |  j6=i |aji | }.
2 3
1 1 1
(14) Is 4 1 3 15 (a) SPD? (b) SPSD? (c) Weakly diagonally dominant?
1 1 3
(15) Using your computer, give 2 Fiedler clusters of the following graphs:

2 3 2 3
3 2 3 3 2 1
(16) Which are SPD? (a) 42 4 25 (b) 42 0 45 (c) Matrix of (9) (d) Matrix of (10)
3 2 3 1 4 5

(17) This is a good time to do the group assignment on Fiedler vector based clustering together
with your group!
(18) Optional: past exam questions on this class
 Class 10: Linear dependence, orthogonality, bases, and rank

(1) Are these vectors independent? If not, give a linear combination that gives 0.
        
1 0 1 2 1 0 1 1 0
(a) 2
, 0
(b) 2
, 5
(c) 2
, 1
(d) 2
, 1
, 1
2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3
1 1 0 1 1 0 1 1 0 1
(e) 4 2 5, 4 2 5, 4 0 5 (f ) 4 2 5, 4 2 5, 4 1 5 (g) 4 2 5, 4 2 5, 4 1 5, 4 15
3 4 1 3 4 0 3 4 0 1
2 3
3 4 1
(2) (a) Determine (by hand) the rank of A = 4 2 3 1 5.
9 5 4
(b) What can you conclude about A being singular or nonsingular ?
(c) What can you conclude about det(A) ?
(d) Are the columns of A independent?
(e) Do the columns of A form a basis for R3 ?
(3) Now change only the 9 in position (3, 1) to 8 and answer the same questions.
 
1 2 1 2
(4) What are the ranks of A = 3 6
and B = 3 6.0001
?
Can you explain from this that determining the rank of a matrix may be a tricky subject
in practical situations?
(5) Determine if the columns of the following matrices:
• form an orthonormal basis,
• form an orthogonal basis (but not orthonormal),
• form a basis (but not orthogonal, let alone orthonormal),
• or do not form a basis at all (are linearly dependent).
  " 1 1
# " 1
p 1
p #
1 1 1 1 2 2
(a) 1 1
(b) 1 1
(c) 2
1
2
1
(d) 2
1
p 2
1
p
2 2 2
2 2
2
2 3 2 3 2 1 1
3 2 1
p 1
p 3
1 1 1 1 0 1 2
0 2 2
2 0 2
2
6 7 6 7
(e) 4 1 0 15 (f ) 4 0 1 05 (g) 4 0 1 05 (h) 4
p
0 1
p
05
1 1 0 1 0 1 1
0 1 1
2 0 1
2
2 2 2 2

(6) Which of the matrices in the previous question

2
are3 orthogonal?
1 0 0
(7) Given is that the columns of the matrix 4↵ 2 05 form an orthogonal basis.
3
Give all options for ↵, , and such that this is possible.
(8) Construct a 4 ⇥ 4 matrix with only ±1 as elements so that the columns are orthogonal.
(9) Is this also possible for a 3 ⇥ 3 matrix? Explain your answer.

↵ ↵
(10) Consider A = ↵ ↵
, where ↵ 2 R. What is:
(a) the maximal and minimal possible rank of A ? (by varying ↵)
(b) the “generic” rank of A ? This means: almost always, except for exceptional cases.
(11) Without looking at the slides, try to deduce why for an orthogonal n ⇥ n matrix Q:
(a) Q has orthonormal columns; (b) Q has orthonormal rows; (c) kQxk = kxk;
(d) \(Qx, Qy) = \(x, y); (e) det(Q) = ±1.
(12) Check (2)(a), (3)(a), and (5)(g,h) on your computer.
(13) Finish the online tests, any overdue homework, and the group work as much as you can.
(14) Optional: past exam questions on this class
 Class 11: Singular Value Decomposition (SVD)

Do all exercises by hand, except for (10) and parts of (1) and (7).

1 1
(1) Determine the singular values of A = 0 1
. Give the exact (algebraic) answer.
What is kAk (= kAk2 ) ? Now check this using the computer.
(2) The singular values of A are the square roots of the eigenvalues of ATA.
To be able to take a square root, the eigenvalues of ATA should be real and 0.
(a) Why are we certain that these are real?
(b) And why that they are 0 ? Hint: Rayleigh quotient.
2 3
 1 
T 2 2 2
(3) Compute by hand the SVD A = U ⌃V of (a) 3
(b) 4 4 5 (c) 2 2
5

(4) kAk2 is as least as large as max | |, the absolute value of the maximal eigenvalue (of A,
not of ATA). Show this by taking the 2-norm of the equation Ax = x.
Hint: We have seen this before in the class on Gershgorin.
(5) Consider the m ⇥ n matrix A, where m n. Suppose we know the SVD A = U ⌃V T .
Explain how we can easily determine the SVD of AT from this. Give a formula, and also
draw a picture such as the one on the slide with the yellow color (“colored areas”).
(6) Explain using formulas why from the SVD A = U ⌃V T of A, we cannot easily get the SVD
of A2 .
2 3
1 2
(7) Consider A = 42 25 and its SVD A = U ⌃V T .
3 3

(a) What do you expect that:

• u1 , the first column of U (the first left singular vector),
• and v1 , the first column of V (the first right singular vector)
will be approximately? Think of the “dominant” column and row, and take care that
they have to have norm 1.
(b) And what do you think that
• v2 , the second column of V (the second singular vector)
will be approximately?
(c) Now compute the SVD on the computer. Is it as expected?
(8) Considering the position of U and V in A = U ⌃V T , how can you remember that the left
singular vectors U give information about the rows, and that the right singular vectors V
give information about the columns ?
(9) From Av = u and ATu = v it follows that ATAv = 2 v. Deduce this.
This gives a nice relation between the singular values of A and the eigenvalues of ATA.

1 2
(10) Let A = 3 4
. Using your computer, compute (a) both eigenvalues of ATA;
(b) both singular values of A; (c) kAk. Explain what you see.
(11) We know that kAk = maxkxk=1 kAxk (as usual, the norm is the 2-norm).
Show using the SVD that kAk = kU ⌃V T k = k⌃k = 1 .
Hint: use the properties of orthogonal matrices of the previous class.
(12) This is a good time to do the group assignment on the term–document (or product–
customer, or keyword–tweet) matrix together with your group!
(13) Optional: past exam questions on this class
 Class 12: Least squares and fitting

(1) (Important question!) Without looking at the slides, show that ATA is symmetric positive
semidefinite (SPSD).
(2) Suppose we have the inconsistent (contradictory) equations x = 8 and x = 10. Solve the
least squares problem minx (x 8)2 + (x 10)2 in the “Calculus way” (set derivative to 0).
This method is always possible but gets trickier and trickier for more variables.
(3) Now solve the above problem in a linear algebra way: set up Ax ⇡ b, and determine
A 2 R2⇥1 and b 2 R2 (so A and b are of the same size here). Then solve x via the normal
equations ATAx = AT b. Is the result natural?
(NB: since here x is a number rather than a vector, we prefer a non-boldface letter.)
(4) Determine by hand the best straight line (of the form y(t) = x1 + x2 t) fitting the points
(0, 0), (1, 1), and (2, 1) by an Ax ⇡ b approach, where A 2 R3⇥2 and b 2 R3 .
Draw a picture to see if your answer makes sense, and compute the sum of squares.
(5) What is the best fit in (4) if one considers a degree-0 (contant) polynomial y(t) = x1 ?
(6) You are an expert in computer graphics, and your boss asks you
to determine a suitable curve (blue line) that approximately fits
the 4 red points. She states that she prefers a simple polyno-
mial of a low degree.
(a) Determine by hand the best quadratic polynomial fitting
these points ( 1, 2), (0, 0), (1, 2), and (2, 28) by a least squares
approach Ax ⇡ b, where A 2 R4⇥3 , b 2 R4 , and x 2 R3 , to
give a curve of the form y(t) = x1 + x2 t + x3 t2 .
(b) Without doing any computations, explain what will happen if we take a cubic polyno-
mial (y(t) = x1 + x2 t + x3 t2 + x4 t3 ) instead.

(7) Check your answer to the previous question by computer, using “fitting” or solve the
normal equations ATAx = AT b for appropriate A and b.
(8) (Important for your understanding and to improve your technical skills!)
Let A be m ⇥ n with m > n. Show that the following statements are equivalent:
(a) A has full rank n
(b) The columns of A are linearly independent
(c) There does not exist a nonzero v with Av = 0
(d) There does not exist a nonzero v with ATAv = 0
(e) ATA is nonsingular
This means that the least squares problem ATAx = AT b has a unique solution if and only
if A is of full rank n.
(9) The best fitting quadratic polynomial through the points ( 1, 1), (0, 0), (1, 1), and (2, 4)
evidently is: . . . . . . (fill the dots!). Now, we consider the “perturbed” points ( 1, 1.1),
(0, 0.1), (1, 0.9), and (2, 4.1). (Here, “perturbed” means: “changed a little bit”.)
Determine by computer the 3 coefficients of the best fitting quadratic polynomial.
Is the answer close to that to the first part of this question?
2 3
1 t1 t21
(10) First show that the determinant of 41 t2 t22 5 is (t2 t1 )(t3 t1 )(t3 t2 ). Conclude that
1 t3 t23
this matrix is nonsingular if the ti ’s are distinct. Then show: if a quadratic polynomial
at2 + bt + c is zero in three di↵erent points, then it must be the zero polynomial!
(11) Optional: past exam questions on this class
 Class 13: Rotations, projections, reflections

1 1 1
(1) (a) Show, using the definition, that P = 2 1 1
is a projection.
(b) Again using the definition, is it also an orthogonal projection?
" 4 4
#
(2) The same questions for P = 7
3
7
3
.
7 7
⇡
(3) Give the 2 ⇥ 2 matrices representing a rotation of 2, ⇡, 32 ⇡, ⇡ 2
2 , 3 ⇡, and 2
3 ⇡.
Which of these is also a reflection in a line? And the reflection in the origin?
(4) Let v 2 R2 be a vector with kvk = 1, and let R = I 2 vvT be the corresponding reflection
across the line orthogonal to v. What do you expect that R2 is (two times reflecting is . . . )?
Now check this algebraically: (I 2 vvT )(I 2 vvT ) = . . . .

cos(↵) sin(↵)
(5) Multiply the rotation matrices sin(↵) cos(↵)
(over angle ↵), and similar over angle .
Use the Calculus formulas sin(↵ + ) = sin(↵) cos( ) + . . . , and a similar one for cos(↵ + ),
to show that the product of these 2 matrices represents the rotation over angle ↵ + .

cos(↵) sin(↵)
(6) What is the inverse of the 2 ⇥ 2 rotation matrix sin(↵) cos(↵)
?

(7) Give the 2 ⇥ 2 matrix for the orthogonal projection onto the line y = 4x.
Test your matrix on two well-chosen vectors (on the axis of projection, and orthogonal to
the axis of projection) to see if it gives the desired result.
(8) Give the 2 ⇥ 2 matrix for the reflection in the line y = 4x.
(9) Give the 2 ⇥ 2 matrix for the oblique projection onto the x-axis along the line y = 2x.

1
Test your matrix on the vector 1
: see if it gives the desired result.

(10) True or false? Show using algebra arguments.

(a) If Q is a 2 ⇥ 2 rotation, then so is QT .
(b) If Q is a 2 ⇥ 2 rotation, then so is Q2 .
(c) If Q is a 2 ⇥ 2 rotation, then Q2 = Q.
(d) The 2 ⇥ 2 identity matrix I2 is a projection.
(e) The 2 ⇥ 2 zero matrix O2 is a projection.
(f ) If P is a 2 ⇥ 2 projection onto a line, then so is P T .
(g) If P is an n ⇥ n projection, then P 3 = P .
(h) If P is an n ⇥ n projection, then P 99 = P .
(i) Every 2 ⇥ 2 projection matrix is nonsingular.
(j) Every 2 ⇥ 2 projection matrix is singular.
(k) If P is a 2 ⇥ 2 orthogonal projection matrix onto a line, then p11 can be < 0.
(11) Show that a 2 ⇥ 2 rotation matrix Q has det(Q) = 1. Why is this natural?
" 1 1
p #
3
(12) Show that 1
2
p 2
1
is a reflection. Across which axis?
2
3 2

(13) Give the 3 ⇥ 3 matrix of a rotation of angle ↵ around the z-axis.

(14) Although this question also holds in the Rn , we consider it in R2 .
Let x, y 2 R2 and let P be an orthogonal projection in the R2 .
(a) Show Pythagoras’ theorem: if x ? y, then kx + yk2 = kxk2 + kyk2 .
Hint: start with (x + y)T (x + y) = . . . . Also make a sketch.
(b) Show that P x ? (I P )x. Hint: start with xT (I P )T P x = . . . .
(c) Use (a) and (b) to show: kxk2 = kP xk2 + k(I P )xk2 . Also sketch a figure.
(d) Conclude that kP xk  kxk. Explain why this logical.
(15) Optional: past exam questions on this class
 Class 14: Splines and truncated SVD

(1) [Splines] Suppose we have a cubic spline on the intervals [0, 1], [1, 2], and [2, 3]. List the
“dofs” (degrees of freedom, number of variables that can/should be chosen), and the number
of requirements.
(2) We are interested in a cubic spline fitting the points ( 1, 1), (1, 0), and (2, 4).
(a) Write down the requirements for the 2 cubic polynomials p (on the interval [ 1, 1]) and
q (on the interval [1, 2]). Add the 2 extra degrees of freedom for p0 ( 1) and q 0 (2).
(b) Formulate a linear system Ax = b, where x contains the coefficients x1 , . . . , x8 .
(c) If time allows, solve x on the computer, and plot the spline; take p0 ( 1) = 1 and
q(2) = 1. (This part (c) is not important for the exam.)
(3) [Truncated SVD] Draw a picture with “boxes” of the sizes of the following matrices in
the (truncated) SVDs, for A with size 10 ⇥ 5:
(a) Draw A = U ⌃V T with ⌃ of size 10 ⇥ 5.
(b) Draw A = U ⌃V T with ⌃ of size 5 ⇥ 5. Explain the di↵erence with (a).
(c) Draw the sizes for the truncated SVD (TSVD) A ⇡ Ak = Uk ⌃k VkT for k = 2.
It is not necessary to give any SVD, just to draw a picture of the sizes.
(4) Suppose A ⇡ Ak = Uk ⌃k VkT is a truncated SVD (TSVD) of A 2 Rm⇥n .
Assume k < min(m, n), so that it is a real truncation.
(a) Are the columns of Uk linearly independent ? The columns of Vk ?
(b) Are the columns of Uk orthogonal, orthonormal ? The columns of Vk ?
(c) Are the columns of Uk a basis for Rm ? The columns of Vk a basis for Rn ?
(d) Is Uk orthogonal? Vk orthogonal?
(5) What is the best rank-1 approximation of the matrices:
  
1 0 2 1 1 1
(a) 0 3
(b) 2 1
(c) 1 1 2 3
1
(6) What is the best rank-1, and the best rank-2 approximation of 4 2 5 ?
3

(7) Suppose you have an image of 200 ⇥ 320 pixels, and every pixel is stored as a real number.
(a) How many bytes are needed to store this image?
(Recall: 1 number takes 64 bits = 8 bytes.)
(b) Suppose we approximate A by A30 , the TSVD with k = 30.
How many bytes are needed now? What are the savings?
(8) Finish all electronic homework tests. This takes time, start early!
(9) Finish all homework.
(10) Ask any questions that you have to the instructors.
(11) Past exams!

GOOD LUCK ON THE EXAM !

See next page for some review / exam training

 Review / exam training

For all questions, give a 2 ⇥ 2 matrix, not equal to a multiple of I or O.

(1) Give a matrix such that A2 = A. What is the name of this type of matrix?
(2) Give a matrix such that A 1 = A, or in other words A2 = I. What type of matrix is an
example of this?
(3) Give a matrix such that A 1 = AT , or in other words AT A = I. What is the name of this
type of matrix?
(4) Give a matrix such that AT = A. What is the name of this type of matrix?
(5) Give a matrix such that AT = A. (What is the name of this type of matrix?)
(6) Give a matrix such that A2 = O. (What is the name of this type of matrix?)
(7) Give a matrix such that A2 = I. What type of matrix is an example of this?
(8) Give a matrix such that A3 = I. What type of matrix is an example of this?
(9) Give a matrix such that A3 = I. What type of matrix is an example of this?
(10) Give a matrix such that A3 = A.
(11) Give 2 properties of a matrix of the form B = AT A.
(12) Name 3 classes (topics) where we have seen a matrix of the form B = AT A.
(13) If we know the eigenvalues and eigenvectors of A, what can we say about those of:
(a) A2 (b) A 1 (c) AT (d) SAS 1
(14) The determinant satisfies det(AB) = det(A) det(B).
Can you give examples of functions f : R ! R that satisfy:
(a) f (xy) = f (x) · f (y) (b) f (xy) = f (x) + f (y) (c) f (x + y) = f (x) · f (y)
(15) Explain to yourself: ", eps, overflow, underflow, cancellation.
(16) Explain to yourself the relations between the notions of:
(non)singular, determinant, inverse matrix, rref, eigenvalues, singular values, independent
columns, independent rows, rank, pivots, solution to homogeneous linear system, solution
to inhomogeneous linear system, basis.
(17) Assign the following words to either “Square only” or “Square and nonsquare”:
determinant, eigenvalue, linear system, least squares problem, matrix norm, SVD, pivot,
rank, orthogonal matrix, projection, reflection, rotation, Laplacian, stochastic matrix.