1.

5 Triangular Factors and Row Exchanges

MATH203-03

Week 2 / TUE

1 / 29
Example
Matrices like
 
  5 0 0 0
" # 2 0 0
1 0 0 0 0 0
, 0 3 0 ,
   
0 2 0 0 1 0
 
0 0 7
0 0 0 6

are said to be diagonal. More generally, a square matrix A is said to be

diagonal if
Aij = 0 whenever i 6= j.

Note
A matrix is diagonal if and only if it is both upper triangular and lower
triangular.

2 / 29
Example
    
a 0 0 0 x 0 0 0 ax 0 0 0
0 b 0 0 0 y 0 0 0 by 0 0 
 
=
 
0 0 c 0  0 0 z 0  0 0 cz 0 
  

0 0 0 d 0 0 0 w 0 0 0 dw

Note
Products of diagonal matrices are diagonal.

3 / 29
Inverses

Note
There is a real number x such that 2x = 1.
For any nonzero a ∈ R, there is x ∈ R such that ax = 1.

Exercise
Show that there is no matrix X such that
   
1 1 1 0
X= .
0 0 0 1

Exercise
Show that there is no matrix X such that
   
1 1 1 0
X = .
0 0 0 1

4 / 29
Definition
If
AX = X A = I,
then X is called an inverse of A, and A is said to be invertible.

Note
If X is an inverse of A, then A is an inverse of X .
To be invertible, a matrix must be square.
If A is n × n, then X is also n × n.
Zero matrix is not invertible.
Every identity matrix is invertible.
Unlike real numbers, a nonzero matrix may not have an inverse.

5 / 29
Theorem
If A is invertible, then its inverse is unique.

Proof
If X and Y are inverses of A, then

X = X I = X (AY ) = (X A)Y = IY = Y .

Notation
The unique inverse of A is denoted by A−1 . Hence,

AA−1 = A−1 A = I.

Exercise
Prove that
(A−1 )−1 = A.

6 / 29
Example
" #" # " #" # " #
1 2 −2 1 −2 1 1 2 1 0
= = ,
3 4 3/2 −1/2 3/2 −1/2 3 4 0 1
so
" #−1 " # " #−1 " #
1 2 −2 1 −2 1 1 2
= and = .
3 4 3/2 −1/2 3/2 −1/2 3 4

Exercise
Show that if A is invertible, then 2A is invertible, and

(2A)−1 = 2−1 A−1 .

Exercise
Show that if A is invertible, then −A is invertible, and

(−A)−1 = −A−1 .
7 / 29
Exercise
Let A and B be square matrices of the same size. Show that if A and B
are invertible, then AB is invertible, and

(AB)−1 = B −1 A−1 .

Exercise
If A and B are invertible, is A + B invertible? Prove your answer.

Exercise
Show that a diagonal matrix is invertible if and only if all its diagonal
entries are nonzero.

Exercise
 −1
1 0 0 0
0 2 0 0
Find  .
 
0 0 3 0


0 0 0 4
8 / 29
Example
    
1 0 0 a1 a2 a3 a1 a2 a3
−2 1 0 b1 b2 b3  = b1 − 2a1 b2 − 2a2 b3 − 2a3 
0 0 1 c1 c2 c3 c1 c2 c3

  
1 0 0 a1 a2 a3
+2 1 0 b1 − 2a1 b2 − 2a2 b3 − 2a3 
0 0 1 c1 c2 c3
   
a1 a2 a3 a1 a2 a3
= b1 − 2a1 + 2a1 b2 − 2a2 + 2a1 b3 − 2a2 + 2a3  = b1 b2 b3 
c1 c2 c3 c1 c2 c3

     
1 0 0 1 0 0 a1 a2 a3 a1 a2 a3
2 1 0 −2 1 0 b1 b2 b3  = b1 b2 b3 
0 0 1 0 0 1 c1 c2 c3 c1 c2 c3

9 / 29
Note
    
1 0 0 1 0 0 1 0 0
a 1 0 −a 1 0 = 0 1 0
    
0 0 1 0 0 1 0 0 1

    
1 0 0 1 0 0 1 0 0
 0 1 0  0 1 0 = 0 1 0
    
b 0 1 −b 0 1 0 0 1

    
1 0 0 1 0 0 1 0 0
0 1 0 0 1 0  0 1 0
=
    
 
0 c 1 0 −c 1 0 0 1

10 / 29
Inverses of Elementary Matrices
 −1  
1 0 0 1 0 0
a 1 0 = −a 1 0
   

0 0 1 0 0 1

 −1  
1 0 0 1 0 0
 0 1 0 =  0 1 0
   
b 0 1 −b 0 1

 −1  
1 0 0 1 0 0
0 1 0 = 0 1 0
   
0 c 1 0 −c 1

Note
E −1 is also lower triangular with 1s in the diagonal.
11 / 29
Forward Elimination via Matrices (Review)

Ax = b E1 Ax = E1 b

E2 E1 Ax = E2 E1 b

E3 E2 E1 Ax = E3 E2 E1 b

(E3 E2 E1 A)x = (E3 E2 E1 b)

Ux = c

Note that
E3 E2 E1 A = U.

12 / 29
Note
     
1 0 0 1 0 0 1 0 0 1 0 0
E3 E2 E1 = 0 1 0  0 1 0 a 1 0 =  a 1 0
     
0 c 1 b 0 1 0 0 1 b + ac c 1

     
1 0 0 1 0 0 1 0 0 1 0 0
E1 E2 E3 = a 1 0  0 1 0 0 1 0 = a 1 0
     
0 0 1 b 0 1 0 c 1 b c 1

Multiplying the elementary matrices in the reverse order (left to

right) allows one to easily read off the entries in the product from the
entries of the elementary matrices.
Every lower triangular matrix with 1s in the diagonal is a product of
elementary matrices.
13 / 29
Note
Let L be a lower triangular matrix with 1s in the diagonal. Suppose that

L = E1 E2 E3 .

Since

(E1 E2 E3 )(E3−1 E2−1 E1−1 ) = (E3−1 E2−1 E1−1 )(E1 E2 E3 ) = I,

we see that
L−1 = (E1 E2 E3 )−1 = E3−1 E2−1 E1−1 .
Note that L−1 is also lower triangular with 1s in the diagonal.

14 / 29
Example
 −1    −1
1 0 0 1 0 0 1 0 0 1 0 0
a 1 0 = a 1 0  0 1 0 0 1 0
b c 1 0 0 1 b 0 1 0 c 1

 −1  −1  −1

1 0 0 1 0 0 1 0 0
= 0 1 0  0 1 0 a 1 0
0 c 1 b 0 1 0 0 1

   
1 0 0 1 0 0 1 0 0
= 0 1 0  0 1 0 −a 1 0
0 −c 1 −b 0 1 0 0 1

 
1 0 0
=  −a 1 0
−b + ac −c 1

15 / 29
Theorem
Every lower triangular matrix with 1s in the diagonal,
 
1 0 0 ··· 0
∗ 1 . . . . . . ... 
 
 
L =  ... ... . . . . . . 0 ,
 
 
∗ ∗ · · · 1 0
 

∗ ∗ ··· ∗ 1

is invertible.

Example
 −1  
1 0 0 1 0 0
2 1 0 = −2 1 0
   
3 4 1 5 −4 1

16 / 29
Theorem
Every upper triangular matrix with 1s in the diagonal,
 
1 ∗ ··· ∗ ∗
0
 1 ··· ∗ ∗ 
 . .. .. .. 
U = 0 . . . . . ,
 
.
. .. ..

. . . 1 ∗


0 ··· 0 0 1

is invertible.

Exercise
Find  −1
1 2 3
0 1 4 .
 
0 0 1

17 / 29
LU Factorization

E3 E2 E1 A = U

E2 E1 A = E3−1 U

E1 A = E2−1 E3 −1 U

A = E1−1 E2−1 E3−1 U

A = LU

18 / 29
Example (Nonsingular Case)
     
1 0 0 1 0 0 1 0 0 2 1 1
0 1 0  0 1 0 −2 1 0A =  0 −8 −2
     
0 1 1 1 0 1 0 0 1 0 0 1

 −1  −1  −1  

1 0 0 1 0 0 1 0 0 2 1 1
A = −2 1 0  0 1 0 0 1 0 0 −8 −2
       
 
0 0 1 1 0 1 0 1 1 0 0 1

    
1 0 0 1 0 0 1 0 0 2 1 1
= 2 1 0  0 1 00 1 0 0 −8 −2
    
0 0 1 −1 0 1 0 −1 1 0 0 1

  
1 0 0 2 1 1
= 2 1 0 0 −8 −2 = LU
  
−1 −1 1 0 0 1
19 / 29
Example (Singular Case)
    
1 1 1 1 0 0 1 1 1
2 2 5  2
= 1 0 0 0 3 = LU
    

4 4 8 4 4/3 1 0 0 0
Note that there are only 2 pivots in U.

Note
LU factorization is not unique for singular matrices:
    
1 1 1 1 0 0 1 1 1
2 2 5 = 2 1 0  0 0 3 (For any x ∈ R.)
    
4 4 8 4 4/3 x 0 0 0

We will see later that LU factorization is unique for nonsingular matrices.

20 / 29
Exercise
Find an LU factorization of
 
1 0 1
2 2 2 .
 
3 4 5

21 / 29
An Application of LU Factorization
Let A = LU. Then
(
Ly = b
Ax = b ⇐⇒ LUx = b
Ux = y

L and U are triangular matrices, so Ly = b and Ux = y are easily

solved by back-substitution.a
Finding the factorization A = LU does require forward elimination,
which involves more computations than back-substitution.
Once A = LU is known, however, only back-substitution is needed to
solve Ax = c for any other c.
(
Ly = c
Ax = c ⇐⇒ LUx = c
Ux = y
a
L is lower triangular, so the back-substitution for Ly = b goes from top to
bottom.
22 / 29
Example
    
2 3 3 1 0 0 2 3 3
0 5 7 = 0 1 00 5 7,
    
6 9 8 3 0 1 0 0 −1
so
    


 1 0 0 y1 2
    
0 1 =
0y2  2
   

 
     

2 3 3 x1 2  3 0 1 y3 5



0 5 7 x2  = 2
    
    
6 9 8 x3 5 


 2 3 3 x1 y1

    
0 5 7 x2  = y2 

    

 

0 0 −1 x3

 y3

23 / 29
LU Factorization with Row Exchanges
Sometimes, the forward elimination requires row exchanges.

(E6 E5 σ3 E4 σ2 E3 E2 E1 σ1 E )A = U

(E60 E50 E40 E30 E20 E10 P)A = U

PA = (E10−1 E20−1 E30−1 E40−1 E50−1 E60−1 )U

PA = LU

It follows that for every matrix A, there exists a permutation matrix P

such that PA has an LU factorization. Then we solve Ax = b as follows:
(
Ly = Pb
Ax = b ⇐⇒ PAx = Pb ⇐⇒ LUx = Pb
Ux = y

24 / 29
Note
 
  a1 a2 a3  
2 0 0 b1 b2 b3  = 2a1 2a2 2a3
c1 c2 c3

 
  a1 a2 a3  
0 3 0 b1 b2 b3  = 3b1 3b2 3b3
c1 c2 c3

 
  a1 a2 a3  
0 0 5 b1 b2 b3  = 5c1 5c2 5c3
c1 c2 c3

    
2 0 0 a1 a2 a3 2a1 2a2 2a3
0 3 0 b1 b2 b3  = 3b1 3b2 3b3 
0 0 5 c1 c2 c3 5c1 5c2 5c3

25 / 29
Note
    
a1 a 2 a 3 2 0 0 a1 /2 a2 /2 a3 /2
b1 b2 b3  = 0 3 0 b1 /3 b2 /3 b3 /3
    
c1 c2 c3 0 0 5 c1 /5 c2 /5 c3 /5

    
2 1 1 2 0 0 1 1/2 1/2
 0 −8 −2 =  0 −8 0 0 1 1/4
    
0 0 1 0 0 1 0 0 1

26 / 29
LDU Factorization (Nonsingular Case)

A = LU

    
2 1 1 1 0 0 2 1 1
 4 −6 0 = −2 1 0  0 −8 −2
    
−2 7 2 1 −1 1 0 0 1

   
1 0 0 2 0 0 1 1/2 1/2
= −2 1 0  0 −8 0 0 1 1/4
   
1 −1 1 0 0 1 0 0 1

= LDU 0

Note that U 0 is upper triangular, with 1s in the diagonal. Note that L,

D, and U 0 are invertible.
27 / 29
Theorem
The LDU factorization of a square nonsingular matrix is unique.

Proof
Let A be a square nonsingular matrix, and suppose that
A = L1 D1 U1 = L2 D2 U2 .
Then
L−1
2 L1 D1 U1 = D2 U2 ⇐⇒ L−1 −1
2 L1 D1 = D2 U2 U1

⇐⇒ L−1 −1 −1
2 L1 = D2 U2 U1 D1
Note the following:
L2−1 L1 is lower triangular with 1s in the diagonal.
D2 U2 U1−1 D1−1 is upper triangular.
Hence, they must both be the identity matrix:
L2 −1 L1 = D2 U2 U1 −1 D1 −1 = I
28 / 29
Proof (cont’d.)
Then L1 = L2 , and
D2 U2 U1−1 D1−1 = I ⇐⇒ D2 U2 U1−1 = D1

⇐⇒ U2 U1−1 = D2−1 D1 .
Note the following:
U2 U1−1 is upper triangular with 1s in the diagonal.
D2−1 D1 is diagonal.
Hence, they must both be the identity matrix:
U2 U1−1 = D2−1 D1 = I
Then U1 = U2 and D1 = D2 .

Exercise
Show that the LU factorization of a square nonsingular matrix is unique.

29 / 29