Spring, 2020 Solving Linear Systems 1

MANAGEMENT
MATHEMATICS
Solving Linear Systems

盧信銘
台大資管
Spring, 2020 Solving Linear Systems 2

Topics
• Preliminaries
• Echelon Form of a Matrix
• Elementary Matrices; Finding A-1
• LU-Factorization

Key Terms:
Row echelon form 階梯矩陣
Reduced row echelon form 簡化階梯矩陣
Spring, 2020 Solving Linear Systems 3

Topics
• Preliminaries
• Echelon Form of a Matrix
• Elementary Matrices; Finding A-1
• LU-Factorization

Key Terms:
Row echelon form 階梯矩陣
Reduced row echelon form 簡化階梯矩陣
Spring, 2020 Solving Linear Systems 4

Preliminaries
• The goal of this lecture:
• Develop systematic procedure for solving linear systems that
works directly with the matrix form
• Establish conditions for a solution to a linear system to exist
• Establish conditions for a unique solution to a linear system
• Re-examine solutions to homogeneous systems
• Develop a systematic procedure for computing A–1
• Discover equivalent conditions to nonsingularity

• Will also examine a special technique for solving linear

systems: LU decomposition

☼
Spring, 2020 Solving Linear Systems 5

Topics

• Preliminaries
• Echelon Form of a Matrix
• Elementary Matrices; Finding A-1
• LU-Factorization

Key Terms:
Row echelon form 階梯矩陣
Reduced row echelon form 簡化階梯矩陣
Spring, 2020 Solving Linear Systems

Echelon Form of a Matrix

• Consider the linear system in matrix form

1 1 2 𝑥1 −1
1 −2 1 𝑥2 = −5
3 1 1 𝑥3 3

• To find the solution, transformed it into:

1 1 2 𝑥 −1
1 1
4 1 0 0 𝑥1 1
0 1 𝑥2 = ⇒ 0 1 0 𝑥2 = 2
3 𝑥3 3 0 0 1 𝑥3 −2
0 0 1 −2
Row echelon form Reduced row echelon form
Row echelon form 階梯矩陣 Reduced row echelon form 簡化階梯矩陣

6 ☼
Spring, 2020 Solving Linear Systems 7

Echelon Form of a Matrix

• We will use the following terminologies.
WIP
Pivot Columns
 1 1 2 −1  1 1 2 −1  1 1 2 −1
 1 −2 1 −5  0 −3 −1 −4   0 −3 −1 −4 
     
3 1 1 3 0 −2 −5 3 0 0 −13 3 26 3

Pivots (主元)

1 1 2 −1  1 0 0 1 
  0 1 1 3 4 3   0 1 0 2 
    ☼
0 0 1 −2  0 0 1 −2 
Spring, 2020 Solving Linear Systems 8

Echelon Form of a Matrix

• Defn - An m x n matrix A is said to be in reduced row echelon form if it
has the following properties
a) If any rows consist of entirely zeros, they are at the bottom of the
matrix (全0的row在最下面)
b) If a row does not consist entirely of zeros, the first nonzero entry
in that row is 1 (不是全0的row，第一個非零元素是1)
c) If rows i and i + 1 are two successive rows which do not consist
entirely of zeros, then the first nonzero entry of row i + 1 is to the
right of the first nonzero entry of row i (上下兩個相鄰的row, 下面
那個row的第一個非零元素在上面那個row的第一個非零元素的
右邊)
d) If a column contains the first nonzero entry of some row, then all
the other entries in that column are zero (如果一個column有某個row
的第一個非零元素，則這個column的其他元素為零)
• Defn - If an m x n matrix A satisfies properties (a), (b) and (c), it is said
to be in row echelon form ☼
Spring, 2020 Solving Linear Systems 9

Echelon Form of a Matrix

Examples - Row Echelon Form

1 5 0 2 −2 4
1 0 0 0
0 1 0 3 4 8
0 1 0 0
0 0 0 1 7 −2 0 0 1 0
0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0

0 0 1 3 5 7 9
0 0 0 0 1 −2 3
0 0 0 0 0 1 2
0 0 0 0 0 0 1
0 0 0 0 0 0 0 ☼
Spring, 2020 Solving Linear Systems 10

Echelon Form of a Matrix

Examples - Reduced Row Echelon Form

1 0 0 0 1 0 0 0 −2 4
0 1 0 0 4 8
0 1 0 0
0 0 0 1 7 −2
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0

1 2 0 0 1
0 0 1 2 3
0 0 0 0 0
☼
Spring, 2020 Solving Linear Systems 11

Your Turn
• True or false: the following matrices are in reduced row
echelon form
• (1) (2)
1 2 0 4 1 0 3 4
0 0 0 0 0 2 −2 5
0 0 1 −3 0 0 1 2
• (3) (4)
0 0 0 1 0 0 1 2 3 4
0 0 0 0 1 0 0 1 −2 5
0 0 0 0 0 1 0 0 1 2
0 0 0 0 0 0 0 0 0 1

☼
Spring, 2020 Solving Linear Systems 12

Echelon Form of a Matrix

Comments - Row Echelon Form

• Can define column echelon form and reduced column
echelon form in a similar manner. Basically, they are just the
transpose of the corresponding row forms
• Close connection between echelon forms and the solution of
linear equations

☼
Spring, 2020 Solving Linear Systems 13

Elementary Row Operation (基本列運算)

• Defn - An elementary row operation on a matrix A is any
one of the following operations
a) Interchange rows i and j of matrix A
b) Multiply row i of A by c ≠ 0
c) Add c times row i of A to row j of A, i ≠ j
• The three row operations are connect to
a) Interchange two equations
b) Multiply a given equation by a nonzero constant
c) Add a constant times one equation to another equation
• We know that the solution to a system of linear equations is
unchanged if we do any of the following
☼
Spring, 2020 Solving Linear Systems 14

Row Equivalence

• Defn - An m x n matrix P is row equivalent to an m x n

matrix Q if P can be obtained by applying a finite sequence
of elementary row operations to Q.
• The basic method for solving the system AX = B is to apply
elementary row operations to the augmented matrix [ A:B ]
to get a new augmented matrix [ C:D ] such that the CX =
D is easy to solve
• If [ C:D ] is in row echelon form, the method is called
Gaussian Elimination
• If [ C:D ] is in reduced row echelon form, the method is
called Gauss-Jordan Elimination
☼
Spring, 2020 Solving Linear Systems 15

What do you think?

• Given a nonzero matrix A, can we perform elementary

row operations to transform A to
• Row echelon form? (true or false)
• Reduced row echelon form? (true or false)
• If your answer is false, can you provide an example?

𝑎11 𝑎12 ⋯ 𝑎1𝑛 0 1 0 0 ⋯ 0

𝑎21 𝑎22 ⋯ 𝑎2𝑛 0 0 1 0 ⋯ 0
𝑎31 𝑎32 ⋯ 𝑎3𝑛 ➔ 0 0 0 1 ⋯ 0
⋮ ⋮ ⋮ ☼
𝑎𝑛1 𝑎𝑛2 ⋯ 𝑎𝑚𝑛 0 0 0 0 ⋯ 0
Spring, 2020 Solving Linear Systems 16

What do you think? (Discussion)

Spring, 2020 Solving Linear Systems 17

Using elementary row operations to Theorem 2.3

solve linear systems

• Since performing elementary row operations on the

augmented matrices correspond to the following operations
on the equations
a)Interchange two equations
b)Multiply a given equation by a nonzero constant
c)Add a constant times one equation to another equation
❑➔Two row equivalent augmented matrices corresponds to
two equitant linear systems (same solutions)
• If the system AX = b has an augmented matrix [ A:b ]
which is row equivalent to an augmented matrix [ C:d ],
then the systems AX = b and CX = d have the same
solution
• A and C are m x n matrices.
Spring, 2020 Solving Linear Systems 18

Your Turn
• True or false: If A and C are row equivalent m x n
matrices, then the homogeneous systems AX = 0 and
CX = 0 are equivalent.
Discussion

QED
Spring, 2020 Solving Linear Systems 19

Back substitution
• If Gauss-Jordan reduction is used, the solution is just read
off. If Gaussian elimination is used, the solution must be
found by back substitution

1 2 3 4 5 6 Note 𝑥5 is the free variable

0 1 2 3 −1 7
[𝑪: 𝑫] =
0 0 1 2 3 7 Set x5 = r and backsolve
0 0 0 1 2 9
𝑥1 = −1
𝑥2 = 2 + 5𝑟
𝑥3 = −11 + 𝑟
𝑥1 + 2𝑥2 + 3𝑥3 + 4𝑥4 + 5𝑥5 =6
𝑥4 = 9 − 2𝑟
𝑥2 + 2𝑥3 + 3𝑥4 − 𝑥5 =7
𝑥3 + 2𝑥4 + 3𝑥5 =7
𝑥4 + 2𝑥5 =9
Infinite number of solutions
☼
Spring, 2020 Solving Linear Systems 20

Identify inconsistent system in row echelon form

Example

1 2 3 4 5
𝐂:𝐃 = 0 1 2 3 6
0 0 0 0 1

The last equation is 0 x1+0 x2+0 x3+0 x4 = 1, which cannot

be satisfied
• Clearly, several things can happen and a careful and
organized analysis of the problem is required

☼
Spring, 2020 Solving Linear Systems 21

Echelon Form of a Matrix

Homogeneous Systems
• Homogeneous system, AX = 0, of m equations in n
unknowns occurs often applications
• Example (1) How many free variables in the solution
of this system?
1 0 0 0 2 0
0 0 1 0 3 0
0 0 0 1 4 0 (2) What does it mean to have free variables?
0 0 0 0 0 0
Spring, 2020 Solving Linear Systems 22

The Case of m<n

Homogeneous Systems
• Theorem - A homogeneous system of m linear equations in n
unknowns always has a nontrivial solution if m < n, i.e. if the
number of unknowns exceeds the number of equations.
• Proof - Let B be a matrix in reduced row echelon form that is
row equivalent to A. The homogeneous systems AX = 0 and
BX = 0 are equivalent. (proof continue on next slide)
A B
𝑎11 𝑎12 ⋯ 𝑎1𝑛 0 1 𝑏12 0 ⋯ 0 0
𝑎2𝑛 𝑎22 ⋯ 𝑎2𝑛 0 0 0 1 ⋯ 0 0
➔
⋱ ⋮ ⋱ ⋮
𝑎𝑚1 𝑎𝑚2 ⋯ 𝑎𝑚𝑛 0 0 0 0 ⋯ 1 0 ☼
Spring, 2020 Solving Linear Systems 23

Echelon Form of a Matrix

Homogeneous Systems
• Proof (continue): Let z be the number of nonzero rows of B.
Then z ≤ m. Since m < n, then z < n. So, we are solving z
equations in n unknowns and can solve for z of the unknowns
in terms of the remaining n - z unknowns, which can take any
values. So, BX = 0 and thus AX = 0 have nontrivial solutions.
A B
𝑎11 𝑎12 ⋯ 𝑎1𝑛 0 1 𝑏12 0 ⋯ 0 0
𝑎2𝑛 𝑎22 ⋯ 𝑎2𝑛 0 0 0 1 ⋯ 0 0
➔
⋱ ⋮ ⋱ ⋮
𝑎𝑚1 𝑎𝑚2 ⋯ 𝑎𝑚𝑛 0 0 0 0 ⋯ 1 0

At most m non-zero rows

☼

QED
Spring, 2020 Solving Linear Systems 24

Topics

• Preliminaries
• Echelon Form of a Matrix
• Elementary Matrices; Finding A-1
• LU-Factorization
Spring, 2020 Solving Linear Systems 25

Elementary Matrices
• Defn - An n x n elementary matrix of Type I, Type II or
Type III is a matrix obtained from the identity matrix In
by performing a single elementary row operation of
Type I, Type II or Type III, respectively

Type I - exchange two rows

Type II - multiply a row by a nonzero constant
Type III - add a multiple of one row to another

☼
Spring, 2020 Solving Linear Systems 26

Your Turn
Which of these are elementary matrices and nonelementary matrices?
1 0 0 1 0 0
(𝑎) 0 3 0 1 0 0
(𝑏) (𝑐) 0 1 0
0 0 1 0 1 0
0 0 0

1 0 0
1 0 1 0 0
(𝑑) 0 0 1 (𝑒)
2 1 (𝑓) 0 2 0
0 1 0 0 0 −1

☼
Spring, 2020 Solving Linear Systems 27

Elementary Matrices
• Have defined three elementary row operations
• Type I - exchange two rows
• Type II - multiply a row by a nonzero constant
• Type III - add a multiple of one row to another

• Each elementary row operation can be performed on a

matrix A by multiplying A by an appropriate elementary
matrix E

• This is useful for proofs, but not for practical computations

since too many operations are required. ☼
Spring, 2020 Solving Linear Systems 28

Elementary Matrices (Type I)

Elementary Row Operations
• Interchange rows i and j of matrix A

0 1 0 𝑎11 𝑎12 𝑎13 𝑎14 𝑎21 𝑎22 𝑎23 𝑎24

1 0 0 𝑎21 𝑎22 𝑎23 𝑎24 = 𝑎11 𝑎12 𝑎13 𝑎14
0 0 1 𝑎31 𝑎32 𝑎33 𝑎34 𝑎31 𝑎32 𝑎33 𝑎34

In general, rows i and j of A can be interchanged by

multiplying A by the m x m matrix E defined as
1 if 𝑘 ≠ 𝑖 and 𝑘 ≠ 𝑗
𝑒𝑝𝑞 = 0 except for 𝑒𝑘𝑘 = ቊ
0 if 𝑘 = 𝑖 or 𝑘 = 𝑗
and 𝑒𝑖𝑗 = 1, 𝑒𝑗𝑖 = 1

☼
E is a special case of a permutation matrix
Spring, 2020 Solving Linear Systems 29

Elementary Matrices (Type II)

Elementary Row Operations

• Multiply row i of A by c ≠ 0

0 1 0 𝑎11 𝑎12 𝑎13 𝑎14 𝑎11 𝑎12 𝑎13 𝑎14

0 𝑐 0 𝑎21 𝑎22 𝑎23 𝑎24 = 𝑐𝑎21 𝑐𝑎22 𝑐𝑎23 𝑐𝑎24
0 0 1 𝑎31 𝑎32 𝑎33 𝑎34 𝑎31 𝑎32 𝑎33 𝑎34

In general, row i of A can be multiplied by c by multiplying A by the m

x m matrix E defined as
0 if 𝑝 ≠ 𝑞
𝑒𝑝𝑞 = ቐ1 if 𝑝 = 𝑞 ≠ 𝑖
𝑐 if 𝑝 = 𝑞 = 𝑖
☼
Spring, 2020 Solving Linear Systems 30

Elementary Matrices (Type III)

Elementary Row Operations
• Add c times row i of A to row j of A, i ≠ j

0 1 0 𝑎11 𝑎12 𝑎13 𝑎14

0 1 0 𝑎21 𝑎22 𝑎23 𝑎24 =
0 𝑐 1 𝑎31 𝑎32 𝑎33 𝑎34
𝑎11 𝑎12 𝑎13 𝑎14
𝑎21 𝑎22 𝑎23 𝑎24
𝑐𝑎21 + 𝑎31 𝑐𝑎22 + 𝑎32 𝑐𝑎23 + 𝑎33 𝑐𝑎24 + 𝑎34

In general, can add c times row i to 𝑐 𝑝 = 𝑗 and 𝑞 = 𝑖

𝑒𝑝𝑞 = ቐ​1 𝑝=𝑞
row j of A by multiplying A by the 0 otherwise ☼
m x m matrix E defined as
Spring, 2020 Solving Linear Systems 31

Additional Comments on Elementary Matrices

Comments
• The appropriate elementary matrix E may be obtained by
performing the desired operation on the rows of the m x m
identity matrix Im
• Can define elementary matrices F to perform elementary
column operations by doing the corresponding operations
on the columns of the identity matrix. The matrices F are
applied by post multiplying, i.e. AF

☼
Spring, 2020 Solving Linear Systems 32

Theorem - Let A be an m x n matrix and let an elementary row

operation of Type I, Type II or Type III be performed on A to
yield matrix B. Let E be the elementary matrix obtained from
Im by performing on it the same elementary row operation that
was performed on A. Then B = EA.
Proof: (Theorem 2.5) Omitted.
Try it yourself! Do a general argument based on
the examples in the previous slides.
This theorem means that the following two things are
essentially the same:
Type I row operation 𝐵 = 𝐸1 𝐴
A→ Type II row operation →B 𝐵 = 𝐸2 𝐴
☼
Type III row operation 𝐵 = 𝐸3 𝐴
Spring, 2020 Solving Linear Systems 33

Elementary Matrices and Row Equivalence

• Theorem 2.6 - If A and B are m x n matrices, then A is
row equivalent to B if and only if 𝑩 = 𝑬𝒌 𝑬𝒌−𝟏 … 𝑬𝟐 𝑬𝟏 𝑨,
𝑘
where 𝑬𝑖 𝑖=1 are m x m elementary matrices.

• Proof omitted (intuitive reasoning below)

• Based on the previous theory, we know that elementary
row operations and the corresponding elementary
matrices have the same effect.
• The above theorem is a formal statement of this fact. ☼
 Spring, 2020 Solving Linear Systems 34

Your Turn
(Using elementary matrices)
Find a sequence of elementary matrices that can be used to write
the matrix A in row-echelon form.
0 1 3 5
𝐴 = 1 −3 0 2
2 −6 2 0

☼
Spring, 2020 Solving Linear Systems 35

Your Turn

☼
Spring, 2020 Solving Linear Systems 36

Ex:
Elementary Matrix Inverse Matrix

0 1 0 0 1 0
𝐸1 = 1 0 0 = 𝑅12 (𝑅12 )−1 = 𝐸1−1 = 1 0 0 = 𝑅12 (Elementary Matrix)
0 0 1 0 0 1

1 0 0 1 0 0
(−2) (−2)
𝐸2 = 0 1 0 = 𝑅13 (𝑅13 )−1 = 𝐸2−1 = 0 1 0 (2)
= 𝑅13 (Elementary Matrix)
−2 0 1 2 0 1

1 0 0 1
(2)
1 0 0
1
0 1 0 (2 ) (𝑅3 )−1 = 𝐸3−1 = 0 1 0 = 𝑅3
(2)
𝐸3 = 1 = 𝑅3 (Elementary Matrix)
0 0 2
0 0
2

☼
 Spring, 2020 Solving Linear Systems 37

Inverse of Elementary Matrices

• Theorem - An elementary matrix is nonsingular and its
inverse is an elementary matrix of the same type.
• Proof -Type I - Let E switch rows i and j. Then the effects of
E may be undone by applying E again, i.e. EE = I. So, E is
invertible and E-1 = E
• Type II - Let E multiply the ith row by c ≠ 0. Let F multiply
the ith row by 1 / c. Then EF = FE = I. So, E is invertible
and E-1 = F
• Type III - Let E add c times the ith row to the jth row. Let F
add (-c) times the ith row to the jth row. Then EF = FE = I.
So, E is invertible and E-1 = F
☼
Nonsingular matrix: 非奇異矩陣
Spring, 2020 Solving Linear Systems 38

Your Turn
1 0 0 0 0
0 1 0 0 0
• What is the inverse of 0 0 1 0 0?
0 71 0 1 0
0 0 0 0 1
Spring, 2020 Solving Linear Systems 39

Ex: From A to I, and finding 𝐴−1

Find a sequence of elementary matrices whose product is

−1 −2
𝐴=
3 8

Sol:
(−1) (−3)
−1 −2 𝑟1 1 2 𝑟12 1 2
𝐴=
3 8 3 8 0 2
1
(2) (−2)
𝑟2 1 2 𝑟21 1 0
=𝐼
0 1 0 1

1
(−2) (2) (−3) (−1)
Therefore 𝑅21 𝑅2 𝑅12 𝑅1 𝐴 =𝐼
☼
Spring, 2020 Solving Linear Systems 40

1
(−2) (2) (−3) (−1)
𝑅21 𝑅2 𝑅12 𝑅1 𝐴 =𝐼
1
(−1) −1 (−3) −1 (2) (−2)
Thus 𝐴 = (𝑅1 ) (𝑅12 ) (𝑅2 )−1 (𝑅21 )−1

(−1) (3) (2) (2)

= 𝑅1 𝑅12 𝑅2 𝑅21

−1 0 1 0 1 0 1 2
=
0 1 3 1 0 2 0 1

☼
Spring, 2020 Solving Linear Systems 41

Computing A-1
• Have shown that if A is nonsingular, then it is row equivalent to In . Also,
have shown that it can be expressed as the product of elementary
matrices.
−1 −1
𝐈𝑛 = 𝐄𝑘 𝐄𝑘−1 ⋯ 𝐄2 𝐄1 𝐀 ⇒ 𝐀 = 𝐄1−1 𝐄2−1 ⋯ 𝐄𝑘−1 𝐄𝑘

−1 −1 −1
• Then 𝑨−1 = 𝑬1−1 𝑬−1
2 ⋯ 𝑬𝑘−1 𝑬𝑘 = 𝑬𝑘 𝑬𝑘−1 ⋯ 𝑬2 𝑬1 So, A-1 can
be represented as the product of the elementary matrices that reduce A
to In
• Create the partitioned matrix 𝑨 𝑰𝑛 and apply the operations that
reduce A to In
☼
𝐄𝑘 𝐄𝑘−1 ⋯ 𝐄2 𝐄1 𝐀 𝐈𝑛 = 𝐈𝑛 𝐀−1
Spring, 2020 Solving Linear Systems 42

Computing A-1
Example 1 1 3
• Compute inverse of 𝐀= 1 2 3
0 1 1
1 1 3 1 0 0 1 1 3 1 0 0
1 2 3 0 1 0 ⇒ 0 1 0 −1 1 0
0 1 1 0 0 1 0 1 1 0 0 1

1 1 3 1 0 0 1 1 0 −2 3 −3
⇒ 0 1 0 −1 1 0 ⇒ 0 1 0 −1 1 0
0 0 1 1 −1 1 0 0 1 1 −1 1

1 0 0 −1 2 −3 −1 2 −3
⇒ 0 1 0 −1 1 0 ⇒ 𝐀−1 = −1 1 0
☼
0 0 1 1 −1 1 1 −1 1
Spring, 2020 Solving Linear Systems 43

Equivalent statements for n x n matrix A

(proof omitted)
1. A is nonsingular
2. AX = 0 has the trivial solution only. (implied by 1.)
3. A is row equivalent to In. (implied by 2.)
4. A is a product of elementary matrices. (implied by 3.)
5. The system AX = B has a unique solution for every n x 1
matrix B (implied by 1.)

☼
Spring, 2020 Solving Linear Systems 44

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

Equivalent statements for n x n matrix A

• A is singular
• AX = 0 has non-trivial solutions. (proof omitted, see
text book)
• A is row equivalent to B with a row of zeros (will
discuss about this later).
Another important property:
• If A and B are n x n matrices such that AB = In , then
BA = In . Thus A is nonsingular and B = A-1. (will
discuss about this later)
Spring, 2020 Solving Linear Systems 45

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

• Theorem - An n x n matrix A is singular if and only if A is row

equivalent to a matrix B that has a row of zeros.
• Consider the following example.
• Given a square matrix A, we can drive it toward reduced row
echelon form by elementary row operations.
• What will we get in the end?
𝑟𝑜𝑤 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛 𝑟𝑜𝑤 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛 𝑟𝑜𝑤 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛
•𝐴 𝐵1 𝐵2 … 𝑍
• Will we get an identity matrix? What are the possibilities?
• Ans: Two cases; Z can be an identify matrix, or not.
• If Z is not an identity matrix, then it must has a row of zeros.
Spring, 2020 Solving Linear Systems 46

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

• Theorem - An n x n matrix A is singular if and only if A

is row equivalent to a matrix B that has a row of zeros.
• Proof -  Let A be singular. Apply Gauss-Jordan
reduction to A to get a matrix B in reduced row
echelon form which is row equivalent to A. B cannot
be In since if it were, A would be nonsingular. Since B
is in reduced row echelon form, B must have at least
one row of zeros at the bottom.
 Let a A be row equivalent to a matrix B that has a
row of all zeros.
(… continue next slide)
☼
 Spring, 2020 Solving Linear Systems 47

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

• Proof  (continued) -
[First show that B is singular]
The matrix B is singular since there does not exist a
matrix C such that BC = CB = In .
To see this, let the ith row of B consist of all zeros. The
ith row of BC is generated by taking the ith row of B
and multiplying each column of C by it. So, the ith row
of BC is all zeros, but BC = In . So, B must be singular.
𝑏11 𝑐11 1
1
0 0 ⋯ 0 ⋯ = 0 0 ⋯ 0
i-th row i-th row
𝑏𝑛𝑛 𝑐𝑛1 𝑐𝑛𝑛 0 1
☼

QED
Spring, 2020 Solving Linear Systems 48

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

• Proof  (continued) -
[Show that B is singular → A is singular]
A is row equivalent to B, so 𝑩 = 𝑬𝒌 𝑬𝒌−𝟏 ⋯ 𝑬𝟐 𝑬𝟏 𝑨

If A is nonsingular then so is B since it is the product

of nonsingular matrices. Since we know B is singular,
then A must be singular.

QED
Spring, 2020 Solving Linear Systems 49

Watch Video:
Finding A-1 https://cool.ntu.edu.tw/courses/
1220/modules/items/34648

• The preceding theorem gives a way to determine if A

is singular. Also, the determination of
singularity/nonsingularity can be made in the process
of computing A-1
• Form the augmented matrix [𝑨|𝑰𝑛 ], then put it into
reduced row echelon form to get 𝑪 𝑫 .
• If C = In then A is nonsingular and D = A-1

• If C ≠ In then C has a row of zeros and thus is

singular. So, A is singular and A-1 does not exist
☼
Spring, 2020 Solving Linear Systems 50

Watch Video: https://cool.ntu.edu.tw/courses/1220/modules/items/34648

• Theorem - If A and B are n x n matrices such that AB = In,

then BA = In . Thus A is nonsingular and B = A-1
• Proof - First show that if AB = In then A is nonsingular. (by contradiction)

• Suppose that A is singular. Then A is row equivalent to a matrix C that

𝑘
has a row of zeros, i.e. 𝐶 = 𝐸𝑘 𝐸𝑘−1 … . 𝐸2 𝐸1 𝐴, where 𝐸𝑖 𝑖=1 is a set of
elementary matrices.
• Since 𝐶𝐵 = 𝐸𝑘 𝐸𝑘−1 … . 𝐸2 𝐸1 𝐴𝐵, so CB is row equivalent to AB.
• Since C has a row of zeros, so does CB.
• AB is singular since it is row equivalent to a matrix with a row of zeros.
• However AB = In , which is nonsingular.
• ➔Contradiction.

• So, A is nonsingular and A-1 exists. A-1 ( AB ) = A-1 In so B = A-1

☼
Spring, 2020 Solving Linear Systems 51

Topics

• Preliminaries
• Echelon Form of a Matrix
• Elementary Matrices; Finding A-1
• LU-Factorization

☼
Spring, 2020 Solving Linear Systems 52

LU-factorization
If the nn matrix A can be written as the product of a lower
triangular matrix L and an upper triangular matrix U, then A=LU
is an LU-factorization of A

𝐴 = 𝐿𝑈 L is a lower triangular matrix

U is an upper triangular matrix

If a square matrix A can be row reduced to an upper triangular

matrix U using only the row operation of adding a multiple of
one row to another row below it,
then it is easy to find an LU-factorization of A.
𝐸𝑘 ⋯ 𝐸2 𝐸1 𝐴 = 𝑈
𝐴
𝐴 = 𝐿𝑈 ☼
Spring, 2020 Solving Linear Systems 53

LU-factorization
Ex: (LU-factorization)
1 2 (𝑏) 𝐴
(𝑎) 𝐴 = 1 −3 0
1 0
= 0 1 3
2 −10 2
Sol: (a)

𝑟
−1)(
1 2 12 1 2
𝐴= =𝑈
1 0 0 −2

(−1)
⇒ 𝑅12 𝐴 = 𝑈

(−1)
⇒ 𝐴 = (𝑅12 )−1 𝑈 = 𝐿𝑈

(−1) (1) 1 0
⇒ 𝐿 = (𝑅12 )−1 = 𝑅12 =
1 1

☼
Spring, 2020 Solving Linear Systems 54

LU-factorization
(b)
1 −3 0 𝑟13
(−2) 1 −3 0 (4)
𝑟23 1 −3 0
𝐴= 0 1 3 0 1 3 0 1 3 =𝑈
2 −10 2 0 −4 2 0 0 14

(4) (−2)
⇒ 𝑅23 𝑅13 𝐴 = 𝑈

(−2) (4)
⇒ 𝐴 = (𝑅13 )−1 (𝑅23 )−1 𝑈 = 𝐿𝑈

(−2) (4) (2) (−4)

⇒ 𝐿 = (𝑅13 )−1 (𝑅23 )−1 = 𝑅13 𝑅23

1 0 0 1 0 0 1 0 0
= 0 1 0 0 1 0 = 0 1 0
2 0 1 0 −4 1 2 −4 1

☼
Spring, 2020 Solving Linear Systems 55

LU-factorization
Solving Ax=b with an LU-factorization of A

𝐴𝑥 = 𝑏 If A = LU , then LUx = b
Let y = Ux, then Ly = b

Two steps:

(1) Write y = Ux and solve Ly = b for y

(2) Solve Ux = y for x

☼
 Spring, 2020 Solving Linear Systems 56

LU-factorization
Ex: (Solving a linear system using LU-factorization)
𝑥1 − 3𝑥2 = −5
𝑥2 + 3𝑥3 = −1
2𝑥1 − 10𝑥2 + 2𝑥3 = −20
Sol:
1 −3 0 1 0 0 1 −3 0
𝐴= 0 1 3 = 0 1 0 0 1 3 = 𝐿𝑈
2 −10 2 2 −4 1 0 0 14

(1) Let 𝑦 = 𝑈𝑥, and solve 𝐿𝑦 = 𝑏

𝑦1 𝑦1 = −5
1 0 0 −5
𝑦2 = −1 ⇒ 𝑦2 = −1
0 1 0
𝑦3 𝑦3 = −20 − 2𝑦1 + 4𝑦2
2 −4 1 −20
= −20 − 2(−5) + 4(−1) = −14
☼
Spring, 2020 Solving Linear Systems 57

LU-factorization
(2) Solve the following system 𝑈𝑥 = 𝑦
1 −3 0 𝑥1 −5
0 1 3 𝑥2 = −1
0 0 14 𝑥3 −14
So
𝑥3 = −1
𝑥2 = −1 − 3𝑥3 = −1 − (3)(−1) = 2
𝑥1 = −5 + 3𝑥2 = −5 + 3(2) = 1

Thus, the solution is

1
𝐱= 2
−1
☼
 Spring, 2020 Solving Linear Systems 58

0 1 2
• Example: Find the LU factorization of 𝐴 = 1 1 1.
2 3 5
• Sol:
(1)
We cannot do 𝑟21 because LU factorization required to perform only the
row operation of adding a multiple of one row to another row below it.

• In this case, we cannot perform LU factorization for A unless switching

two rows first.
0 1 0
• Use 𝑃 = 1 0 0 , a permutation matrix, to switch the first two rows:
0 0 1
1 1 1
𝑃𝐴 = 0 1 2 .
2 3 5
• Now we can perform LU factorization on 𝑃𝐴 ☼
Spring, 2020 Solving Linear Systems 59

• Example Sol. (continued):

• 𝑃𝐴 =
1 1 1 1 1 1 1 1 1
(−2) −1
0 1 2 →𝑟13 → 0 1 2 →𝑟23 → 0 1 2 = 𝑈;
2 3 5 0 1 3 0 0 1
−1 −2
• 𝑅23 𝑅13 𝑃𝐴 = 𝑈 →
1 0 0 1 0 0 1 0 0 1 1 1
2 1
• 𝑃𝐴 = 𝑅13 𝑅23
𝑈= 0 1 0 0 1 0 𝑈= 0 1 0 0 1 2
2 0 1 0 1 1 2 1 1 0 0 1
0 1 0 1 0 0 1 1 1
−1
• 𝐴 = 𝑃 𝐿𝑈 = 1 0 0 0 1 0 0 1 2
0 0 1 2 1 1 0 0 1

☼
Spring, 2020 Solving Linear Systems 60

Your Turn

• Find an LU factorization of the matrix 𝐴 =

1 1 1 1
1 2 3 4
.
1 3 6 10
1 4 10 20
Spring, 2020 Solving Linear Systems 61

Your Turn
1 1 1 1
1 2 3 4
𝐴= .
1 3 6 10
1 4 10 20
Spring, 2020 Solving Linear Systems 62

The Global Positioning System (GPS)

• GPS is used in a variety of situations for determining
geographical locations.
• Very useful for civilian and military use.
• How does it work?
• It solves a linear system using the information received
from four satellites.
Spring, 2020 Solving Linear Systems 63

Z: through North Pole

GPS
• To simplify the discussion, set the radii of
the earth to 1.
• The surface of earth is:
𝑥2 + 𝑦2 + 𝑧2 = 1 x
• Time unit: 1/100 seconds y
• Light speed = 0.47
• Idea: three-dimensional triangulation.
• Complication: We do not know the distance
to the satellites!
Spring, 2020 Solving Linear Systems 64

GPS (Cont’d.)
• Need very precise clock to be able to compute the
distance from the receiver to the satellites!
• This type of clock is very expensive.
• Need a cheap solution
• → Solve for distance using one additional satellite.
•
Satellite Position Time (from
midnight)
1 (1.11, 2.55, 2.14) 1.29
2 (2.87, 0.00, 1.43) 1.31
3 (0.00, 1.08, 2.29) 2.75
4 (1.54, 1.01, 1.23) 4.06
 Spring, 2020 Solving Linear Systems 65

GPS (Cont’d.)
• Let (𝑥, 𝑦, 𝑧) be your position, and let 𝑡 be the signals arrive.
• What is the distance from you to the first satellite?
• 𝑑 = 0.47 (𝑡 − 1.29)
•𝑑= 𝑥 − 1.1 2 + 𝑦 − 2.55 2 + 𝑧 − 2.14 2
• Putting the two equations together and rearrange,
• 𝑥 − 1.11 2 + 𝑦 − 2.55 2 + 𝑧 − 2.14 2 = 0.472 𝑡 − 1.29 2
• ➔
• 2.22𝑥 + 5.10𝑦 + 4.28𝑧 − 0.57𝑡 = 𝑥 2 + 𝑦 2 + 𝑧 2 − 0.22𝑡 2 + 11.95
Equation (1)
Satellite Position Time (from
midnight)
1 (1.11, 2.55, 2.14) 1.29
2 (2.87, 0.00, 1.43) 1.31
 Spring, 2020 Solving Linear Systems 66

GPS (Cont’d.)
• Repeat the process for all satellites
• 2.22𝑥 + 5.10𝑦 + 4.28𝑧 − 0.57𝑡 = 𝑥 2 + 𝑦 2 + 𝑧 2 − 0.22𝑡 2 + 11.95
• 5.74𝑥 + 2.86𝑧 − 0.58𝑡 = 𝑥 2 + 𝑦 2 + 𝑧 2 − 0.22𝑡 2 + 9.90
• 2.16𝑦 + 4.58𝑧 − 1.21𝑡 = 𝑥 2 + 𝑦 2 + 𝑧 2 − 0.22𝑡 2 + 4.74
• 3.08𝑥 + 2.02𝑦 + 2.46𝑧 − 1.79𝑡 = 𝑥 2 + 𝑦 2 + 𝑧 2 − 0.22𝑡 2 + 1.26
• Subtract the first equation from each of the other equations:
• 3.52𝑥 − 5.10𝑦 − 1.42𝑧 − 0.01𝑡 = −2.05
• −2.22𝑥 − 2.94𝑦 + 0.30𝑧 − 0.64𝑡 = −7.21
• 0.86𝑥 − 3.08𝑦 − 1.82𝑧 − 1.22𝑡 = −10.69
Spring, 2020 Solving Linear Systems 67

GPS (Cont’d.)
• The augmented matrix:

3.52 −5.10 −1.42 −0.01 −2.05

• −2.22 −2.94 0.30 −0.64 −7.21 −→
0.86 −3.08 −1.82 −1.22 −10.69
1 0 0 0.36 2.97
• 0 1 0 0.03 0.81 −→
0 0 1 0.79 5.91
• 𝑥 = 2.97 − 0.36𝑡
• 𝑦 = 0.81 − 0.03𝑡
• 𝑧 = 5.91 − 0.79𝑡
• Substitute to Equation (1)
Spring, 2020 Solving Linear Systems 68

GPS (Cont’d.)
• 2.97 − 0.36𝑡 − 1.11 2 + 0.81 − 0.03𝑡 − 2.55 2 +
5.91 − 0.79𝑡 − 2.14 2 = 0.472 𝑡 − 1.29 2
• ➔ 0.54𝑡 2 − 6.65𝑡 + 20.32 = 0
• ➔ 𝑡 = 6.74 and 𝑡 = 5.56
• The first solution gives 𝑥, 𝑦, 𝑧 = (0.55, 0.61, 0.56)
• The second solution gives 𝑥, 𝑦, 𝑧 = 0.96, 0.65, 1.46 . (not
on surface, rejected)
• ➔ 𝑥, 𝑦, 𝑧 = (0.55, 0.61, 0.56)
Spring, 2020 Solving Linear Systems 69

Homework
• Watch Video:
https://cool.ntu.edu.tw/courses/1220/modules/items/3464
8
• 歡迎在影片中留言。

• 2.1: 5, 8
• 2.2: 13, 15, 28
• 2.3: 11, 26, 28
• 2.5: 9

• Computer Homework: GPS (gps hw_v2.html)