Algebra
Orthogonal diagonalization
• A matrix A is said to be orthogonally diagonalizable if there is
an orthogonal matrix P and a diagonal matrix D such that:
• D=PTAP
• where the elements in the diagonal of the matrix D are the
eigenvalues of the matrix A.
Theorem
• The following conditions are equivalent for an n×n matrix A.
• 1. A has an orthonormal set of n eigenvectors.
• 2. A is orthogonally diagonalizable.
• 3. A is symmetric.
If A has an orthonormal set of n eigenvectors, A is
orthogonally diagonalizable.
• (1) ⇔ (2) (If 1, then 2. If 2, then 1.)
• Let x1, x2, ..., xn be orthonormal set of n eigenvectors of A. Then, P =
[x1 x2 ... xn] is an orthogonal matrix.
• Since the columns of P are orthogonal and hence independent, P is
invertible.
• A is diagonalizable if and only if it has eigenvectors x1, x2, . . . , xn
such that the matrix P=[x1 x2 . . . xn] is invertible(Theorem i).
• By theorem i, A is diagonalizable. The diagonal matrix D=P-1AP.
• Since P is an orthogonal matrix, P-1=PT.
• Hence, D=P-1AP= PTAP i.e A is orthogonally diagonalizable.
If A is orthogonally diagonalizable, A has an
orthonormal set of n eigenvectors,
• Suppose that A is orthogonally diagonalizable. Then there exists an
orthogonal matrix P such that PTAP is a diagonal matrix. If P has
columns {x1,x2,…xn}, then B = {x1,x2, …xn} is a set of n orthonormal
vectors in Rn.
• Since B is a set of n orthonormal vectors, B is independent. Let PTAP =
diag(l1,l2,….ln) = D (Here li are the eigenvalues of A).
• Then AP = PD, so
• Thus Axi = lixi for each i, 1≤ I ≤ n, implying that B consists of eigenvectors
of A. Therefore, A has an orthonormal set of n eigenvectors.
If A is orthogonally diagonalizable, A is
symmetric.
• (2)⇒ (3) (If 2, then 3)
• Since A is orthogonally diagonalizable, let D=PTAP be the diagonal
matrix, where P-1 =PT.
• Then, A = PDPT.
• But DT = D because the transpose of a diagonal matrix is the diagonal
matrix itself.
• So, this gives AT =(PT)TDTPT = PDPT = A
• Hence, A is symmetric.
If A is symmetric, A is orthogonally
diagonalizable.
• Third,
• (3)⇒ (2) (If 3,then 2)
• We need an intermediate theorem to prove this.
• Lemma
• Suppose that λ1 is an eigenvalue of the nxn symmetric matrix A, with
corresponding unit eigenvector v1. Then there is an orthogonal matrix
P whose 1st column is v1, such that
• where A1 is an (n-1)x (n-1) symmetric matrix and O is a zero matrix.
• Proof:
• First, we know that we can extend the set {v1} to a basis {v1,
…….,vn} for Rn. And then we can use the Gram-Schmidt
Procedure on this basis to get an orthonormal basis {w1,
…….,wn} for Rn. But, since
• v1 is already a unit vector, the Gram-Schmidt Procedure will
set w1= v1, so
• B = { v1, w2, ……wn} is an orthonormal basis for Rn. Let P =[ v1
w2..... wn].
• Since λ1 is a unit eigenvector of v1, we have that v1. Av1 = λ1
v1. v1= λ1 .
• So we've shown that (PTAP)11 =1. Since B is orthonormal, we
get that w1 .Av1= w1 .( λ1 v1)= λ1 w1. v1 = 0.
• So (PTAP)i1 = 0 for all 2≤ i ≤ n.
• And the fact that (PTAP) is symmetric tells us that (PTAP)1i= 0 =
0 for all 2 ≤ i ≤ n. And
• This means that PTAP=
• where A1 must be symmetric because PTAP is symmetric
Proof
• The proof is by induction on n, the size of our symmetric matrix A.
The base case is when n = 1, which means A = [a], and A is diagonalized
by the orthogonal matrix P = [1] to PTAP = [1][a][1] = [a].
• For our induction step, suppose we know that all (n-1)x(n-1) symmetric
matrices are orthogonally diagonalizable, and let A be an nxn
symmetric matrix.
• Let λ be the eigenvalue of A with corresponding unit eigenvector v1.
• By Lemma, we know that there is an orthogonal matrix Q=[v1 w2....wn] such that
QTAQ=
where A' is an (n-1)x(n-1) symmetric matrix.
• By our inductive hypothesis, there is an (n-1)x (n-1) orthogonal matrix Q' and an
(n-1) x (n - 1) diagonal matrix D' such that Q'TA'Q'= D'.
• Also, A′=Q′D′Q′T .
• Hence,
• It follows that
• We obtained a diagonal matrix in the middle but we can't yet say that
A has been orthogonally diagonalized.
• However, if we let U= , D= , we obtain A=UDUT
• If we can show that U is an orthogonal, A becomes orthogonally
diagonalizable.
• Indeed, it is so. Thus, A=UDUT (orthogonally diagonalizable).
• If 1,then 2.
• If 2,then 1.
• If 2,then 3.
• If 3,then 2.
• Therefore,
• If 1,then 3.
• If 3,then 1.
Theorem: The Principal Axes Theorem
• Let A be a n x n symmetric matrix. Then there is an orthogonal change
of variable, x = Py, that transforms the quadratic form xTAx, into a
quadratic form yTDy with no cross-product term.
• Suppose we have a quadratic form
• Q= xTAx where xT = [x x ...x ]
, 1 2 n (*)
• Here A is symmetric. So, we know that we can find an orthogonal
matrix P (i.e., PT = P-1) that diagonalizes A,
(i.e. P-1AP = D = PTAP).
• If we replace A with PDPT in (*) then we obtain
• Q = xT(PDPT)x
= (xTP)D(PTx)
• Now, if we define a new set of vector yT= [y1 y2 ...yn] via the
substitution x= Py then the previous line becomes
• Q = ((Py)TP)D(PTPy)
• = yT (PTP) D (PTP) y
= yT D y = λ1y12 + λ2 y22 +…+ λn yn2
• There are no cross product terms here.
• Sources
• https://people.math.carleton.ca/~kcheung/math/notes/MAT
H1107/wk10/10_orthogonal_diagonalization_proof.html
• LINEAR ALGEBRA with Applications (W. Keith Nicholson)
• Thank you!
