EE731 Lecture Notes: Matrix Computations

for Signal Processing

James P. Reilly c _
Department of Electrical and Computer Engineering
McMaster University
September 24, 2006
0 Preface
This collection of ten chapters of notes will give the reader an introduc-
tion to the fundamental principles of linear algebra for application in many
disciplines of modern engineering and science, including signal processing,
control theory, process control, applied statistics, robotics, etc. We assume
the reader has an equivalent background to a freshman course in linear alge-
bra, some introduction to probability and statistics, and a basic knowledge
of the Fourier transform.
The rst chapter, some fundamental ideas required for the remaining portion
of the course are established. First, we look at some fundamental ideas of
linear algebra such as linear independence, subspaces, rank, nullspace, range,
etc., and how these concepts are interrelated.
In chapter 2, the most basic matrix decomposition, the socalled eigendecom-
position, is presented. The focus of the presentation is to give an intuitive
insight into what this decomposition accomplishes. We illustrate how the
1
eigendecomposition can be applied through the Karhunen-Loeve transform.
In this way, the reader is made familiar with the important properties of
this decomposition. The Karhunen-Loeve transform is then generalized to
the broader idea of transform coding. The ideas of autocorrelation, and the
covariance matrix of a signal, are discussed and interpreted.
In chapter 3, we develop the singular value decomposition (SVD), which is
closely related to the eigendecomposition of a matrix. We develop the rela-
tionships between these two decompositions and explore various properties
of the SVD.
Chapter 4 deals with the quadratic form and its relation to the eigendecom-
position, and also gives an introduction to error mechanisms in oating point
number systems. The condition number of a matrix, which is a critical part
in determining a lower bound on the relative error in the solution of a system
of linear equations, is also developed.
Chapters 5 and 6 deal with solving linear systems of equations by Gaussian
elimination. The Gaussian elimination process is described through a bigger
block matrix approach, that leads to other useful decompositions, such as the
Cholesky decomposition of a square symmetric matrix.
Chapters 710 deal with solving leastsquares problems. The standard least
squares problem and its solution are developed in Chapter 7. In Chapter
8, we develop a generalized pseudoinverse approach to solving the least
squares problem. The QR decomposition in developed in Chapter 9, and its
application to the solution of linear least squares problems is discussed in
Chapter 10.
Finally, in Chapter 11, the solution of Toeplitz systems of equations and its
underlying theory is developed.
2
1 Fundamental Concepts
The purpose of this lecture is to review important fundamental concepts in
linear algebra, as a foundation for the rest of the course. We rst discuss the
fundamental building blocks, such as an overview of matrix multiplication
from a big block perspective, linear independence, subspaces and related
ideas, rank, etc., upon which the rigor of linear algebra rests. We then
discuss vector norms, and various interpretations of the matrix multiplication
operation. We close the chapter with a discussion on determinants.
1.1 Notation
Throughout this course, we shall indicate that a matrix A is of dimension
m n, and whose elements are taken from the set of real numbers, by the
notation A R
mn
. This means that the matrix A belongs to the Cartesian
product of the real numbers, taken mn times, one for each element of A.
In a similar way, the notation A C
mn
means the matrix is of dimension
mn, and the elements are taken from the set of complex numbers. By the
matrix dimension mn, we mean A consists of m rows and n columns.
Similarly, the notation a R
m
(C
m
) implies a vector of m elements which
are taken from the set of real (complex) numbers. When referring to a single
vector, we use the term dimension to denote the number of elements.
Also, we shall indicate that a scalar a is from the set of real (complex)
numbers by the notation a R(C). Thus, an upper case bold character
denotes a matrix, a lower case bold character denotes a vector, and a lower
case non-bold character denotes a scalar.
By convention, a vector by default is taken to be a column vector. Further,
for a matrix A, we denote its ith column as a
i
. We also imply that its jth
row is a
T
j
, even though this notation may be ambiguous, since it may also be
taken to mean the transpose of the jth column. The context of the discussion
will help to resolve the ambiguity.
3
1.2 Fundamental Linear Algebra
1.2.1 Vector Spaces
Formally, a vector space is dened as follows:
A vector space o satises two requirements:
1. If x and y are in o, then x +y is still in o.
2. If we multiply any vector x in o by a scalar c, then cx is still in o.
This denition implies that if a set of vectors are in a vector space, then any
linear combination of these vectors are also in the space. We now expand on
this denition of a vector space, as follows.
Suppose we have a set of vectors [a
1
, . . . , a
n
], where a
i
R
m
, i = 1, . . . , n,
and a set of scalars c
i
R, i = 1, . . . , n. Then the vector y R
m
dened by
y =
n

i=1
c
i
a
i
(1)
is referred to as a linear combination of the vectors a
i
. (Note that in this
section and in the sequel, all column vectors are assumed to be of length m,
unless stated otherwise).
We wish to see if the above equation can be represented as a matrixvector
multiplication, which has the advantage of being more compact. We note
that each coecient c
i
in (1) multiplies all elements in the corresponding
vector a
i
.
y =
_

_
. . .
_

_
A
_

_
c
1
c
2
.
.
.
c
n
_

_
c
(2)
4
Consider the accompanying diagram, depicting matrixvector multiplication,
where each vertical line represents an entire column a
i
of A. In a manner
similar to (1), from the rules of matrixvector multiplication, we see that
each element c
i
of c multiples only elements of the corresponding column a
i
;
i.e., coecient c
i
interacts only with the column a
i
. Thus, (1) can be written
in the form
y = Ac (3)
where A R
mn
= [a
1
, . . . , a
n
], and c R
n
= [c
1
, . . . , c
n
]
T
.
Instead of using (3) to dene a single vector, we can use it to dene a set of
vectors, which we will denote as o. Consider the expression
o = y R
m
[y = Ac, c R
n
(4)
where now it is implied that c takes on all possible values within R
n
. The
set o dened in this way is referred to as a vector space, and is the set of all
linear combinations of the vector set. The dimension of the vector space is
the number of independent directions that span the space; e.g., the dimension
of the universe is 3.
The dimension of the vector space o
_
denoted as dim(o)
_
is not necessarily
n, the number of vectors or columns of A. In fact, dim(o) n. The quantity
dim(o) depends on the characteristics of the vectors a
i
. For example, the
vector space dened by the vectors a
1
and a
2
in Fig. 1 below is the plane of
the paper. The dimension of this vector space is 2:

B
a
1
a
2
Figure 1: A vector set containing two vectors.
If a third vector a
3
which is orthogonal to the plane of the paper were added
to the set, then the resulting vector space would be the threedimensional
universe. A third example is shown in Figure 2. Here, since none of the
vectors a
1
. . . , a
3
have a component which is orthogonal to the plane of the
5

B
E
a
1
a
2
a
3
Figure 2: A vector set containing three vectors.
paper, all linear combinations of this vector set, and hence the corresponding
vector space, lies in the plane of the paper. Thus, in this case, dim(o) is 2,
even though there are three vectors in the set.
In this section we have dened the notion of a vector space and its dimension.
Note that the term dimension when applied to a vector is the number of
elements in the vector. The term length is also used to indicate the number
of elements of a vector.
1.2.2 Linear Independence
A vector set [a
1
, . . . , a
n
] is linearly independent under the conditions
n

j=1
c
j
a
j
= 0 if and only if c
1
, . . . , c
n
= 0 (5)
This means that means that a set of vectors is linearly independent if and
only if the only zero linear combination of the vectors has coecients which
are all zero.
Let o be the vector space corresponding to the vector set [a
1
, . . . , a
n
]. This
set of n vectors is linearly independent if and only if dim(o) = n. If
dim(o) < n, then the vectors are linearly dependent. Note that a set of
vectors [a
1
, . . . , a
n
], where n > m cannot be linearly independent. Further, a
linearly dependent vector set can be made independent by removing vectors
from the set.
Example 1
6
A = [a
1
a
2
a
3
] =
_
_
1 2 1
0 3 1
0 0 1
_
_
(6)
This set is linearly independent. On the other hand, the set
B = [b
1
b
2
b
3
] =
_
_
1 2 3
0 3 3
1 1 2
_
_
(7)
is not. This follows because the third column is a linear combination of the
rst two. (1 times the rst column plus 1 times the second equals the
third column). Thus, the coecients c
j
in (5) resulting in zero are any scalar
multiple of (1, 1, 1).
1.2.3 Span, Range, and Subspaces
In this section, we explore these three closely-related ideas. In fact, their
mathematical denitions are almost the same, but the interpretation is dif-
ferent for each case.
Span:
The span of a vector set [a
1
, . . . , a
n
], written as span[a
1
, . . . , a
n
], is the vector
space o corresponding to this set; i.e.,
span [a
1
, . . . , a
n
] =
_
y R
m
[ y =
n

j=1
c
j
a
j
, c
j
R
_
= o. (8)
Subspaces
A subspace is a subset of a vector space. More precisely, a kdimensional
subspace | of o = span[a
1
, . . . , a
n
] is determined by span[a
i
1
, . . . a
i
k
], where
the indices satisfy i
1
, . . . , i
k
1, . . . , n.
7
Note that [a
i1
, . . . a
ik
] is not necessarily a basis for the subspace S. This set
is a basis only if it is a maximally independent set. This idea is discussed
shortly. The set a
i
need not be linearly independent to dene the span or
subset.
For example, the vectors [a
1
, a
2
] in Fig. 1 dene a subspace (the plane of the
paper) which is a subset of the threedimensional universe R
3
.
What is the span of the vectors [b
1
, . . . , b
3
] in example 1?
Range:
The range of a matrix A R
mn
, denoted R(A), is the vector space satisfying
R(A) = y R
m
[ y = Ax, for x R
n
. (9)
Thus, we see that R(A) is the vector space consisting of all linear combi-
nations of the columns a
i
of A, whose coecients are the elements x
i
of x.
Therefore, R(A) span[a
1
, . . . , a
n
]. The distinction between range and span
is that the argument of range is a matrix, while for span it is a set of vectors.
If the columns of A are (not) linearly independent, then R(A) will (not) span
n dimensions. Thus, the dimension of the vector space R(A) is less than or
equal to n. Any vector y R(A) is of dimension (length) m.
Example 3:
A =
_
_
1 5 3
2 4 3
3 3 3
_
_
(the last column is the average of the rst two) (10)
R(A) is the set of all linear combinations of any two columns of A.
In the case when n < m (i.e., A is a tall matrix), it is important to note
that R(A) is indeed a subspace of the m-dimensional universe R
m
. In this
case, the dimension of R(A) is less than or equal to n. Thus, R(A) does not
span the whole universe, and therefore is a subspace of it.
8
1.2.4 Maximally Independent Set
This is a vector set which cannot be made larger without losing indepen-
dence, and smaller without remaining maximal; i.e. it is a set containing the
maximum number of independent vectors spanning the space.
1.2.5 A Basis
A basis for a subspace is any maximally independent set within the subspace.
It is not unique.
Example 4. A basis for the subspace S spanning the rst 2 columns of
A =
_
_
1 2 3
3 3
3
_
_
, i.e., S = span
_
_
_
_
1
0
0
_
_
,
_
_
2
3
0
_
_
_
_
is
e
1
= (1, 0, 0)
T
e
2
= (0, 1, 0)
T
.
1
or any other linearly independent set in span[e
1
, e
2
].
Any vector in S is uniquely represented as a linear combination of the basis
vectors.
1.2.6 Orthogonal Complement Subspace
If we have a subspace S of dimension n consisting of vectors [a
1
, . . . , a
n
], a
i

R
m
, i = 1, . . . , n, for n m, the orthogonal complement subspace S

of S of
dimension mn is dened as
S

=
_
y R
m
[y
T
x = 0 for all x S
_
(11)
1
A vector e
i
is referred to as an elementary vector, and has zeros everywhere except
for a 1 in the ith position.
9
i.e., any vector in S

is orthogonal to any vector in S. The quantity S

is
pronounced Sperp.
Example 5: Take the vector set dening S from Example 4:
S
_
_
1 2
0 3
0 0
_
_
(12)
then, a basis for S

is
_
_
0
0
1
_
_
(13)
1.2.7 Rank
Rank is an important concept which we will use frequently throughout this
course. We briey describe only a few basic features of rank here. The idea
is expanded more fully in the following sections.
1. The rank of a matrix A (denoted rank(A)), is the maximum number of
linearly independent rows or columns in A. Thus, it is the dimension
of R(A). The symbol r is commonly used to denote rank; i.e., r =
rank(A).
2. if A = BC, and r
1
= rank(B), r
2
= rank(C), then rank(A) min(r
1
, r
2
).
3. A matrix A R
mn
is said to be rank decient if its rank is less than
min(m, n). Otherwise, it is said to be full rank.
4. If A is square and rank decient, then det(A) = 0.
5. It can be shown that rank(A) = rank(A
T
). More is said on this point
later.
A matrix is said to be full column (row) rank if its rank is equal to the
number of columns (rows).
10
Example 6: The rank of A in Example 4 is 3, whereas the rank of A in
Example 3 is 2.
Example 7: Consider the matrix multiplication C R
mn
= AB, where
A R
m2
and B R
2n
, depicted by the following diagram:
_

_
_

_
C
=
_

_
_

_
A
_
x x x x
x x x x
_
B
. (14)
Then, the rank of C is at most two. To see this, we realize from our discussion
on the relation between matrix multiplication and the operation of forming
linear combinations that the ith column of C is a linear combination of the
two columns of A whose coecients are the ith column of B. Thus, all
columns of C reside in the vector space R(A). If the columns of A and the
rows of B are linearly independent, then the dimension of this vector space
is two, and hence rank(C) = 2. If the columns of A or the rows of B are
linearly dependent, then rank(C) = 1. This example can be extended in an
obvious way to matrices of arbitrary size.
1.2.8 Null Space of A
The null space N(A) of A is dened as
N(A) = x R
n
,= 0 [ Ax = 0 . (15)
From previous discussions, the product Ax is a linear combination of the
columns a
i
of A, where the elements x
i
of x are the corresponding coe-
cients. Thus, from (15), N(A) is the set of nonzero coecients of all zero
linear combinations of the columns of A. If the columns of A are linearly
independent, then N(A) = by denition, because there can be no coef-
cients except zero which result in a zero linear combination. In this case,
the dimension of the null space is zero, and A is full column rank. The null
11
space is empty if and only if A is full column rank, and is nonempty when
A is column rank decient. Note that any vector in N(A) is of dimension
n. Any vector in N((A)) is orthogonal to the rows of A, and is thus in the
orthogonal complement of the span of the rows of A.
Example 8: Let A be as before in Example 3. Then N(A) = c(1, 1, 2)
T
,
where c R.
A further example is as follows. Take 3 vectors [a
1
, a
2
, a
3
] where a
i
R
3
, i =
1, . . . , 3, that are constrained to lie in a 2dimensional plane. Then there
exists a zero linear combination of these vectors. The coecients of this linear
combination dene a vector x which is in the nullspace of A = [a
1
, a
2
, a
3
].
In this case, we see that A is rank decient.
Another important characterization of a matrix is its nullity. The nullity of
A is the dimension of the nullspace of A. In Example 6 above, the nullity of
A is one. We then have the following interesting property:
rank(A) + nullity(A) = n. (16)
1.3 Four Fundamental Subspaces of a Matrix
The four matrix subspaces of concern are: the column space, the row space,
and their respective orthogonal complements. The development of these four
subspaces is closely linked to N(A) and R(A). We assume for this section
that A R
mn
, r min(m, n), where r = rankA.
1.3.1 The Column Space
This is simply R(A). Its dimension is r. It is the set of all linear combinations
of the columns of A. Any vector in R(A) is of dimension m.
12
1.3.2 The Orthogonal Complement of the Column Space
This may be expressed as R(A)

, with dimension m r. It may be shown

to be equivalent to N(A
T
), as follows: By denition, N(A
T
) is the set x
satisfying:
_

_
_

_
A
T
_

_
x
1
.
.
.
x
m
_

_
= 0, (17)
where columns of A are the rows of A
T
. From (17), we see that N(A
T
) is
the set of x R
m
which is orthogonal to all columns of A (rows of A
T
). This
by denition is the orthogonal complement of R(A). Any vector in R(A)

is of dimension m.
1.3.3 The Row Space
The row space is dened simply as R(A
T
), with dimension r. The row space
is the range of the rows of A, or the subspace spanned by the rows, or the
set of all possible linear combinations of the rows of A. Any vector in R(A
T
)
is of dimension n.
1.3.4 The Orthogonal Complement of the Row Space
This may be denoted as R(A
T
)

. Its dimension is n r. This set must be

that which is orthogonal to all rows of A: i.e., for x to be in this space, x
must satisfy
rows
of
A
_

_
.
.
.
_

_
_

_
x
1
.
.
.
x
n
_

_
= 0. (18)
13
Thus, the set x, which is the orthogonal complement of the row space satis-
fying (18), is simply N(A). Any vector in R(A
T
)

is of dimension n.
We have noted before that rank(A) = rank(A
T
). Thus, the dimension
of the row and column subspaces are equal. This is surprising, because it
implies the number of linearly independent rows of a matrix is the same as
the number of linearly independent columns. This holds regardless of the
size or rank of the matrix. It is not an intuitively obvious fact and there is
no immediately obvious reason why this should be so. Nevertheless, the rank
of a matrix is the number of independent rows or columns.
1.4 Bigger-Block Interpretations of Matrix Multi-
plication
In this section, we take a look at matrix multiplication from the viewpoint
that columns of a matrix product are linear combinations of the columns
of the rst matrix. To standardize our discussion, let us dene the matrix
product C as
C
mn
= A
mk
B
kn
(19)
The three interpretations of this operation now follow:
1.4.1 Inner-Product Representation
If a and b are column vectors of the same length, then the scalar quantity
a
T
b is referred to as the inner product of a and b. If we dene a
T
i
R
k
as
the ith row of A and b
j
R
k
as the jth column of B, then the element c
ij
of
C is dened as the inner product a
T
i
b
j
. This is the conventional small-block
representation of matrix multiplication.
14
1.4.2 Column Representation
This is the next biggerblock view of matrix multiplication. Here we look at
forming the product one column at a time. The jth column c
j
of C may be
expressed as a linear combination of columns a
i
of A with coecients which
are the elements of the jth column of B. Thus,
c
j
=
k

i=1
a
i
b
ij
, j = 1, . . . , n. (20)
This operation is identical to the innerproduct representation above, except
we form the product one column at a time. For example, if we evaluate
only the pth element of the jth column c
j
, we see that (20) degenerates into

k
i=1
a
pi
b
ij
. This is the inner product of the pth row and jth column of A
and B respectively, which is the required expression for the (p, j)th element
of C.
1.4.3 OuterProduct Representation
This is the largestblock representation. Let us dene a column vector a
R
m
and a row vector b
T
R
n
. Then the outer product of a and b is an mn
matrix of rank one and is dened as ab
T
.
Now let a
i
and b
T
i
be the ith column and row of A and B respectively. Then
the product C may also be expressed as
C =
k

i=1
a
i
b
T
i
. (21)
By looking at this operation one column at a time, we see this form of
matrix multiplication performs exactly the same operations as the column
representation above. For example, the jth column c
j
of the product is
determined from (21) to be c
j
=

k
i=1
a
i
b
ij
, which is identical to (20) above.
15
1.4.4 Matrix Multiplication Again
Here we give an alternate interpretation for matrix multiplication by compar-
ing this operation to that of forming linear combinations. Consider a matrix
A premultiplied by B to give Y = BA. (A and B are assumed to have
conformable dimensions). Let us assume Y 1
mn
. Then we can interpret
this operation in two ways:
Each column y
i
, i = 1, . . . , n of Y is a linear combination of the columns
of B, whose coecients are the ith column a
i
of A; i.e.,
y
i
=
n

k=1
b
k
a
ki
= Ba
i
(22)
This operation is very similar to the column representation for matrix
multiplication.
Each row y
T
j
, j = 1, . . . , m of Y is a linear combination of the rows of
A, whose coecients are the jth row of B; i.e.,
y
T
j
=
m

k=1
b
jk
a
T
k
= b
T
j
A. (23)
Using this idea, can matrix multiplication be cast in a row representa-
tion format?
A further related idea is as follows. Consider an orthonormal matrix
Q of appropriate dimension. We know that multiplication of a vector
by an orthonormal matrix results in a rotation of the vector. The
operation QA rotates each column of A. The operation AQ rotates
each row.
1.5 Vector Norms
A vector norm is a means of expressing the length or distance associated with
a vector. A norm on a vector space R
n
is a function f, which maps a point in
16
R
n
into a point in R. Formally, this is stated mathematically as f : R
n
R.
The norm has the following properties:
1. f(x) 0 for all x R
n
.
2. f(x) = 0 if and only if x = 0.
3. f(x +y) f(x) + f(y) for x, y R
n
.
4. f(ax) = [a[f(x) for a R, x R
n
.
We denote the function f(x) as [[x[[.
The p-norms: This is a useful class of norms, generalizing on the idea of
the Euclidean norm. They are dened by
[[x[[
p
= ([x
1
[
p
+[x
2
[
p
+ . . . +[x
n
[
p
)
1/p
. (24)
If p = 1:
[[x[[
1
=

i
[x
i
[
which is simply the sum of absolute values of the elements.
If p = 2:
[[x[[
2
=
_

i
x
i
2
_1
2
= (x
T
x)
1
2
which is the familiar Euclidean norm.
If p = :
[[x[[

= max
i
[x
i
[
which is the largest element of x. This may be shown in the following way.
As p , the largest term within the round brackets in (24) dominates all
17
the others. Therefore (24) may be written as
[[x[[

= lim
p
_
n

i=1
x
p
i
_1
p
= lim
p
[x
p
k
]
1
p
= x
k
(25)
where k is the index corresponding to the largest element x
i
.
Note that the p = 2 norm has many useful properties, but is expensive
to compute. Obviously, the 1 and norms are easier to compute, but
are more dicult to deal with algebraically. All the pnorms obey all the
properties of a vector norm.
1.6 Determinants
Consider a square matrix A R
mm
. We can dene the matrix A
ij
as
the submatrix obtained from A by deleting the ith row and jth column
of A. The scalar number det(A
ij
) ( where det() denotes determinant) is
called the minor associated with the element a
ij
of A. The signed minor
c
ij

= (1)
j+i
det(A
ij
) is called the cofactor of a
ij
.
The determinant of A is the m-dimensional volume contained within the
columns (rows) of A. This interpretation of determinant is very useful as we
see shortly. The determinant of a matrix may be evaluated by the expression
det(A) =
m

j=1
a
ij
c
ij
, i (1 . . . m). (26)
or
det(A) =
m

i=1
a
ij
c
ij
, j (1 . . . m). (27)
Both the above are referred to as the cofactor expansion of the determinant.
Eq. (26) is along the ith row of A, whereas (27) is along the jth column. It
is indeed interesting to note that both versions above give exactly the same
number, regardless of the value of i or j.
18
Eqs. (26) and (27) express the m m determinant detA in terms of the
cofactors c
ij
of A, which are themselves (m 1) (m 1) determinants.
Thus, m 1 recursions of (26) or (27) will nally yield the determinant of
the mm matrix A.
From (26) it is evident that if A is triangular, then det(A) is the product
of the main diagonal elements. Since diagonal matrices are in the upper
triangular set, then the determinant of a diagonal matrix is also the product
of its diagonal elements.
Properties of Determinants
Before we begin this discussion, let us dene the volume of a parallelopiped
dened by the set of column vectors comprising a matrix as the principal
volume of that matrix.
We have the following properties of determinants, which are stated without
proof:
1. det(AB) = det(A) det(B) A, B R
mm
.
The principal volume of the product of matrices is the product of prin-
cipal volumes of each matrix.
2. det(A) = det(A
T
)
This property shows that the characteristic polynomials
2
of A and A
T
are identical. Consequently, as we see later, eigenvalues of A
T
and A
are identical.
3. det(cA) = c
m
det(A) c R, A R
mm
.
This is a reection of the fact that if each vector dening the principal
volume is multiplied by c, then the resulting volume is multiplied by
c
m
.
4. det(A) = 0 A is singular.
This implies that at least one dimension of the principal volume of the
corresponding matrix has collapsed to zero length.
2
The characteristic polynomial of a matrix is dened in Chapter 2.
19
5. det(A) =

m
i=1

i
, where
i
are the eigen (singular) values of A.
This means the parallelopiped dened by the column or row vectors
of a matrix may be transformed into a regular rectangular solid of the
same m dimensional volume whose edges have lengths corresponding
to the eigen (singular) values of the matrix.
6. The determinant of an orthonormal
3
matrix is 1.
This is easy to see, because the vectors of an orthonormal matrix are
all unit length and mutually orthogonal. Therefore the corresponding
principal volume is 1.
7. If A is nonsingular, then det(A
1
) = [det(A)]
1
.
8. If B is nonsingular, then det(B
1
AB) = det(A).
9. If B is obtained from A by interchanging any two rows (or columns),
then det(B) = det(A).
10. If B is obtained from A by by adding a scalar multiple of one row to
another (or a scalar multiple of one column to another), then det(B) =
det(A).
A further property of determinants allows us to compute the inverse of A.
Dene the matrix

A as the adjoint of A:

A =
_

_
c
11
. . . c
1m
.
.
.
.
.
.
c
m1
. . . c
mm
_

_
T
(28)
where the c
ij
are the cofactors of A. According to (26) or (27), the ith row
a
T
i
of

A times the ith column a
i
is det(A); i.e.,

a
T
i
a
i
= det(A), i = 1, . . . , m. (29)
It can also be shown that

a
T
i
a
j
= 0, i ,= j. (30)
3
An orthonormal matrix is dened in Chapter 2.
20
Then, combining (29) and (30) for i, j 1, . . . , m we have the following
interesting property:

AA = det(A)I, (31)
where I is the m m identity matrix. It then follows from (31) that the
inverse A
1
of A is given as
A
1
= [det(A)]
1

A. (32)
Neither (26) nor (32) are computationally ecient ways of calculating a de-
terminant or an inverse respectively. Better methods which exploit the prop-
erties of various matrix decompositions are made evident later in the course.
21