= `Pyxis` ;

di d display di K 1 , Gi , color = "LightSkyBlue", axes = boxed, scaling = constrained, title

= `Pyxis` ;
end do:

> display en , a n , b n , dn , Ordo, Abs ;

Pyxis
20

10

K10

K20
K10 0 10 20

> display en , a n , b n , dn , Ordo, Abs , color = "LightBlue", axes = boxed, scaling = constrained, title
= `Pyxis` ;

L����� A������
Math -UA 9140

L������ N����

Joachim L�������

S����� 2023

Tuesday 20th June, 2023

C�������

Greek Letters and their correspondance in the Latin alphabet . . . . . . . . . . . . . . v

I Matrices Calculus 1

1 Matrices Calculus 2
1.1 De�nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Particular Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Operations on Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
First Operation: Transpose of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Second Operation: Multiplication of a Matrix by a Scalar . . . . . . . . . . . . . . . . . 4
Third Operation: Addition of two Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 5
Fourth Operation: Multiplication of two Matrices . . . . . . . . . . . . . . . . . . . . . 5
1.5 The ring of Square Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 Invertible Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Linear Systems 12
2.1 Linear System as a Matrix Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Elementary Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Gaussian Elimination & Row Reduced Echelon Matrices . . . . . . . . . . . . . . . . . . 16
2.4 Application of Gaussian Elimination to Inversion of Matrices . . . . . . . . . . . . . . . 21
2.5 Application of Gaussian Elimination to LU Factorization . . . . . . . . . . . . . . . . . 23
LU Factorization of Regular Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
P A = LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6 Application of Gaussian Elimination to solving Linear System . . . . . . . . . . . . . . 31
Results for general linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Results About Square Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

II Vector Spaces & Linear Maps 40

3 Vector Spaces 41
3.1 Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
De�nition & Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Vector Subpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
De�nition & Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Intersection and Sum of Vector Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Intersection of Vector subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Sum of Vector subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Supplementary Vector Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4 Spanning Family & Linearly Independent Family of Vectors . . . . . . . . . . . . . . . . 49

ii
L����� A������ Lecture Notes C�������

3.5 Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

De�nition and characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Structure of L(E, F ) and L(E). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Kernel and Image of a linear map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.6 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Basis and Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.7 Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
De�nition & Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Structure of the Set of solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4 Vector Spaces with �nite Dimension 60

4.1 First Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2 Dimension of Vector Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Complete a Family of Vectors into a Basis . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.3 Link Between Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Dimension and Isomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Dimension of a Product of Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Dimension of a Sum of Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4 Rank-Nullity Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Rank of Family of Vectors, of a Linear Map . . . . . . . . . . . . . . . . . . . . . . . . . 66
Rank-Nullity Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Linear Maps and Finite Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5 The four fundamental subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5 Use of Basis 70
5.1 Presentation of the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 Matrix Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Fondamental Formulae . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
A Particular kind of linear maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Rank of a Linear Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Change of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
The case of vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
The case of Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
The case of Endomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

III Multilinear Maps, Determinants 82

6 Multilinear Maps & Determinants 83

6.1 De�nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 Properties of Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.3 Expanding a Determinant Along a Row or a Column . . . . . . . . . . . . . . . . . . . 86
6.4 Computing a Determinant in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

IV Diagonalization 89

7 Diagonalization 90
7.1 Eigenvalues of an Endomorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

iii S����� 2023

L����� A������ Lecture Notes C�������

De�nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Applications to the Calculation of Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . 92
7.2 Diagonalizable Endomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.3 Diagonalizable Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
De�nition and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Diagonalization in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

V Multilinear Algebra 99

8 Isometries 100

9 Euclidean Spaces 101

VI Applications 102

VII Appendix 104

A Mapping, Image and Preimage 105

A.1 Maps & Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
A.2 Injectivity, Surjectivity and Bijectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Bibliography 111

Index 112

List of Symbols 113

iv S����� 2023
Greek Letters and their correspondance
in the Latin alphabet

Greek letters
Latin letters Classical use in Math
Lowercase letter Capital letter Name
– A alpha a
— B beta b
“ gamma g
” delta d
Á E epsilon e Á: in�nity small
’ Z zeta z
÷ H eta e
◊ theta th ◊: angle
ÿ I iota i
Ÿ K kappa k
⁄ lambda l
µ M mu m
‹ N nu n
› xi x
o O omicron o
r universal cst: fi ¥ 3.141592653589793
fi pi p r
: product
fl P rho r
q q
‡ sigma s : Sum
· T tau tau
‚ Y upsilon u
„, Ï phi f
‰ X khi kh
 psi ps
Ê omega o

v
Part I

1
C������ 1

Matrices Calculus

Throughout this course K denotes either the set of real numbers, denoted R or the set of complex
numbers, denoted C. Moreover, p and q will denote two positive integers.

1.1 De�nition

De�nition 1.1.1 (Matrix)

A p ◊ q matrix with coe�cients in K is a rectangular array of numbers of K with p rows and q
columns. The coe�cients of the matrix are also called terms or elements of the matrix.

We denote M(p, q, K) or Mp,q (K) the set of all p ◊ q matrices with coe�cients in K. In the case
where p = q, we use the shorthand notation Mp (K).

1.2 Notations
In order to represent a p ◊ q matrix A, we usually use the following notation

à í
a1,1 a1,2 · · · a1,q
a2,1 a2,2 · · · a2,q
A := .. .. .. .
. . .
ap,1 ap,2 · · · ap,q

Remark 1.2.1
It is important to respect the convention on the order of indices: we denote ai,j the coe�cient of the
matrix A which is on the ith row and j th column.

When one wants to give the coe�cients of A with a formula, one can also use a condensed notation:

A := (ai,j )16i6p,16j6q or A := (ai,j )(i,j)œJ1,pK◊J1,qK .

where J1, pK stands for {1, 2, · · · , p} for any p in Nú .

2
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Example 1.2.2
Denote A := (i ≠ j)16i62,16j63 . It is clear that:
Ç å
0 ≠1 ≠2
(i ≠ j)16i62,16j63 = .
1 0 ≠1

When there is no ambiguity because we do know the “dimensions” p and q of the matrix A, we simply
write A := (ai,j ). In particular, we will denote (0), and one will call null matrix, the matrix �r which
all coe�cients equal 0. Hence there is a null matrix for every couple of integers (p, q). They are all
denoted the same way. It does not provoke any confusion unless we do not take care to the context.

1.3 Particular Matrices

When p = 1, one says that A is row-matrix. In this case we write:

A = (a1 , a2 , · · · , aq ).

When q = 1, one says that A is column-matrix. In this case we write:

à í
a1
a2
A := .. .
.
ap

When p = q, one says that A is a square matrix of order p.

Let A := (ai,j ) be a square matrix of order p. The diagonal of the matrix A is the line constituted with
the p coe�cients a1,1 , a2,2 , · · · ap,p . The coe�cients on the diagonal are also walled diagonal coe�cients
of A.
A diagonal matrix is a square matrix for which all the non-diagonal terms equal 0. In other words
ai,j = 0 for all couple (i, j) such that i ”= j. This matrix can therefore be written under the form:
â ì
a1 0 ··· 0
.. ..
0 a2 . .
A := .. .
.. ..
. . . 0
0 ··· 0 ap

One also writes A = diag(a1 , a2 , · · · ap ).

An upper triangular (resp. lower triangular) matrix is a square matrix A := (ai,j ) for which all the
terms that are below (resp. above) the diagonal equal 0. In other words ai,j = 0 for all i > j (resp. for
all i < j). Such a matrix can be written under the following form:
â ì â ì
a1,1 a1,2 · · · a1,q a1,1 0 ··· 0
.. .. ..
0 a2,2 · · · . a2,1 a2,2 . .
.. , (resp. .. ).
.. .. ..
. . . ap≠1,p . 0 .
0 ··· 0 ap,p ap,1 · · · ap≠1,p ap,p

3 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

1.4 Operations on Matrices

We will say that two matrices A := (ai,j )(i,j)œJ1,pK◊J1,qK and B := (bi,j )(i,j)œJ1,pÕ K◊J1,qÕ K are equal if
and only if they have the same “dimensions” p and q and the same coe�cients. In other words, if:

’(i, j) œ {1, · · · , p} ◊ {1, · · · , q}, ai,j = bi,j .

Example 1.4.1
De�ne the following matrices
Ç å Ç å
2 ≠2 3 ≠1 2 3
A := & B := .
4 3 7 9 ≠5 6
It is clear that A ”= B since a1,1 ”= b1,1 .

First Operation: Transpose of a Matrix

De�nition 1.4.2 (Transpose of a Matrix)

Let A := (ai,j )(i,j)œJ1,pK◊J1,qK be a matrix with coe�cients in K.
The transpose matrix of A, denoted tA, is the matrix of Mq,p (K) de�ned, for all (i, j) in J1, pK◊J1, qK,
by the following formula:
(tA)i,j := aj,i .

Example 1.4.3
Ö è
2 4
The transpose of the matrix A, de�ned in Example 1.4.1, is: ≠2 3 .
3 7

We easily see that t(tA) = A.

Second Operation: Multiplication of a Matrix by a Scalar

De�nition 1.4.4 (Product of a Matrix by a Scalar)

The product of a matrix A := (ai,j )(i,j)œJ1,pK◊J1,qK by a scalar ⁄ in K is the matrix of Mp,q (K),
denoted ⁄ · A, which is de�ned by the following formula:

(⁄ · A)i,j := ⁄ · ai,j .

Example 1.4.5
If we still consider Matrix A, de�ned at Example 1.4.1, we have:
Ç å Ç å
2 ≠2 3 4 ≠4 6
2·A=2· = .
4 3 7 8 6 14

4 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Let ⁄ be a real number. We have t(⁄ · A) = ⁄ · tA. Moreover, one can have ⁄ · A = (0) if and only if
⁄ = 0 or if A = (0).

Third Operation: Addition of two Matrices

De�nition 1.4.6 (Addition of two Matrices)

The sum of two matrices A := (ai,j )(i,j)œJ1,pK◊J1,qK and B := (bi,j )(i,j)œJ1,pK◊J1,qK of Mp,q (K) is the
matrix of Mp,q (K), denoted A + B, and de�ned by the following formula:

(A + B)i,j = ai,j + bi,j .

Example 1.4.7
Having in mind matrices A and B, de�ned at Example 1.4.1, we can write:
Ç å Ç å Ç å Ç å
2 ≠2 3 ≠1 2 3 2≠1 ≠2 + 2 3 + 3 1 0 6
A+B = + = = .
4 3 7 9 ≠5 6 4 + 9 3≠5 7 + 6 13 ≠2 13

For ay real numbers ⁄ and µ, it is easy to see that the following rules hold.

(⁄ + µ) · A = ⁄ · A + µ · A, ⁄ · (A + B) = ⁄ · A + ⁄ · B, (A + B) = tA + tB.
t
(4.1)
The readers who are familiar with Group Theory noticed that (Mp,q (K), +) is an abelian group.

Fourth Operation: Multiplication of two Matrices

De�nition 1.4.8 (Multiplication of two Matrices)

The product of a matrix A := (ai,j )(i,j)œJ1,pK◊J1,qK by a matrix B := (bk,l )(k,l)œJ1,qK◊J1,rK is the matrix
C := (ci,l )(i,l)œJ1,pK◊J1,rK de�ned, for all (i, l) in J1, pK ◊ J1, rK, by the following formula:
q
ÿ
ci,l = ai,k bk,l . (4.2)
k=1

Example 1.4.9
We hence have:
Ö è
Ç å ≠1 9 Ç å
2 ≠2 3 2 · (≠1) + (≠2) · 2 + 3 · 3 2 · 9 + (≠2) · (≠5) + 3 · 6
· 2 ≠5 =
4 3 7 4 · (≠1) + 3 · 2 + 7 · 3 4 · 9 + 3 · (≠5) + 7 · 6
3 6
Ç å Ç å
≠2 ≠ 4 + 9 18 + 10 + 18 3 46
= = .
≠4 + 6 + 21 36 ≠ 15 + 42 23 63

Remark 1.4.10
In order to de�ne the product of a matrix A by a matrix B, we need that the number of columns of
A equals the number of rows of B. The matrix AB has as much rows as A and as much columns as
B.

5 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Example 1.4.11
Let x1 , x2 , y1 and y2 be real numbers. Let’s compute the two following products.
Ç å Ç å Ç å
y1 y1 y1 x1 y1 x2
(x1 , x2 ) · = (x1 y1 + x2 y2 ), and · (x1 , x2 ) = .
y2 y2 y2 x1 y2 x2

Let ⁄ be a real number. It is easy to see that the following rule hold.

⁄ · (AB) = (⁄ · A) · B = A · (⁄ · B). (4.3)

For the remainder of this section denote r and s two positive integers.

Proposition 1.4.12
The product of two matrices is an associative law. More precisely, if A is a matrix in Mp,q (K), B is
a matrix in Mq,r (K) and C is a matrix in Mr,s (K) then:

(AB) C = A (BC).

Proof 1.4.13
We know that AB belongs to Mp,r (K), and then (AB) C belongs to Mp,s (K). We also know that
BC belongs to Mq,s (K) and then A (BC) belongs to Mp,s (K). Let us show that, for all (i, j) in
J1, pK ◊ J1, sK:
((AB) · C)i,j = (A · (BC))i,j
For all (i, j) in J1, pK ◊ J1, sK, one can write:
r r
Ç q å q
r ÿ
ÿ ÿ ÿ ÿ
((AB) · C)i,j = (AB)i,k ck,j = ai,¸ b¸,k ck,j = ai,¸ b¸,k ck,j
k=1 k=1 ¸=1 k=1 ¸=1

and
q q Ç r
å q ÿ
r q
r ÿ
ÿ ÿ ÿ ÿ ÿ
(A · (BC))i,j = ai,¸ (BC)¸,j = ai,¸ b¸,k ck,j = ai,¸ b¸,k ck,j = ai,¸ b¸,k ck,j ,
¸=1 ¸=1 k=1 ¸=1 k=1 k=1 ¸=1

which achieves the proof. ⇤

Proposition 1.4.14
For any matrices A in Mp,q (K) and B in Mq,r (K), we have:
t
(AB) = tB tA.

Proof 1.4.15
Let A := (ai,j )(i,j)œJ1,pK◊J1,qK and B := (bi,j )(i,j)œJ1,qK◊J1,rK , one can write, for all (i, j) in J1, pK ◊

6 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

J1, rK,
q
ÿ
( (AB))i,j = (AB)j,i =
t
aj,k bk,i
k=1
&
q
ÿ q
ÿ q
ÿ
( B A)i,j =
t t t
B i,k
t
A k,j
= bk,i aj,k = aj,k bk,i ,
k=1 k=1 k=1

which achieves the proof. ⇤

Proposition 1.4.16
The product of matrices is distributive over addition. More precisely, if A and B are matrices of
Mp,q (K) and C and D are matrices of Mq,r (K), then

A · (C + D) = A · C + A · D and (A + B) · C = A · C + B · C.

Proof 1.4.17
It is clear that A · (C + D), A · C and A · D, (A + B) · C and B · C all belong to Mp,r (K). Moreover,
for all (i, j) œ J1, pK ◊ J1, rK, one can write, from one hand:
n
ÿ n
ÿ
(A · (C + D))i,j = ai,k (C + D)k,j = ai,k (ck,j + dk,j )
k=1 k=1

and
n
ÿ n
ÿ n
ÿ
(A · C + A · D))i,j = (A · C)i,j + (A · D)i,j = ai,k ck,j + ai,k dk,j = ai,k (ck,j + dk,j )
k=1 k=1 k=1

and, form the other hand,

n
ÿ n
ÿ
((A + B) · C)i,j = (A + B)i,k ck,j = (ai,k + bi,k ) ck,j
k=1 k=1

and
n
ÿ n
ÿ n
ÿ
(A · C + B · C)i,j = (A · C)i,j + (B · C)i,j = ai,k ck,j + bi,k ck,j = (ai,k + bi,k ) ck,j ,
k=1 k=1 k=1

which achieves the proof. ⇤

De�nition 1.4.18 (Nilpotent Matrix)

A square non-null matrix in Mn (K), denoted N , is called nilpotent if N k = 0Mn (K) , for some
positive integer k.

7 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Example 1.4.19
De�ne the following matrices:
Ö è
Ç å 3 9 ≠9
0 1
C := , D := 2 0 0 .
0 0
3 3 ≠3

It is clear that both A and B are nilpotent since:

Ö è
Ç å 0 0 0
2 0 0
C = & D3 = 0 0 0 .
0 0
0 0 0

1.5 The ring of Square Matrices

From the previous propositions we deduce that the set (Mp (K), +, ·) of square matrices of order p,
endowed with the addition and the multiplication of matrices is a ring1 . Its multiplicative identity
element is the identity matrix of order p, denoted Ip (also denoted I when there is no risk of confusion
about the value of p), and de�ned by setting:
â ì
1 0 ··· 0
.. .
0 1 . ..
Ip := (”i,j )16i,j6p := .. . . ,
..
. . . 0
0 ··· 0 1

where ”i,j denotes the Kronecker symbol which value is 1 if i = j and 0 otherwise. One can easily
check that:
Ip · A = A · Ip = A,
for every p ◊ p matrix A.
1
A ring is a set R equipped with two binary operations, denoted + and ·, satisfying the following three sets of axioms,
called the ring axioms:
1. (R, +) is an abelian group. In other words:

(i) (a + b) + c = a + (b + c), for all a, b, c in R (that is, + is associative).

(ii) a + b = b + a, for all a, b in R (that is, + is commutative).
(iii) There is an element 0R in R such that a + 0 = a, for all a in R (that is, 0R is the additive identity).

2. R is a monoid under multiplication, meaning that:

(i) (a · b) · c = a · (b · c), for all a, b, c in R (that is, · is associative).

(ii) There is an element 1R in R such that a · 1R = a and 1R · a = a, for all a in R (that is, 1R is the multiplicative
identity).

3. Multiplication is distributive over addition, meaning that:

(i) a · (b + c) = (a · b) + (a · c) for all a, b, c in R (left distributivity).

(ii) (b + c) · a = (b · a) + (c · a) for all a, b, c in R (right distributivity).

8 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Remark 1.5.1
1. Note that (Mp (K), ·) is not commutative if p > 1. Hence we have:
Ç å Ç å Ç å Ç å Ç å Ç å
1 0 1 1 1 1 1 1 1 0 1 2
· = and · = .
0 2 0 1 0 2 0 1 0 2 0 2

2. Moreover (Mp (K)+, ·) is not an integral domain. That means that one can �nd two non
zero matrices such that their product equals the null matrix. Hence:
Ç å Ç å Ç å
0 1 0 1 0 0
· = = 0M2 (K) .
0 0 0 0 0 0

Let’s �nish this section by a result on multiplication of matrices, the proof of which can be performed
by induction, on the integer n.

Proposition 1.5.2 (Binomial Formula for Matrices)

When two matrices, denoted S et T , commute, the Binomial theorem applies and allows us to write:
n
Ç å
ÿ n k
’n œ N , (S + T ) =
ú n
S · T n≠k , (5.4)
k=0
k
Ç å
n n!
where := holds for all n > 1 and all 0 6 k 6 n.
k k!(n ≠ k)!

Remark 1.5.3
One can apply the previous result to compute An , once A as been expanded as a sum of a diagonal
matrix D and a nilpotenta matrix N .
a
i.e. there exists a positive integer q such that N q = 0.

Proof 1.5.4
This is nothing but the Binomial Formula, which applies when the two elements commute. ⇤

1.6 Invertible Matrices

De�nition 1.6.1 (Invertible Matrix)

A matrix A of Mp (K) is said to be invertible, if there exists a matrix B in Mp (K) such that:

B · A = A · B = Ip .

The matrix B is denoted A≠1 and is called invert of the matrix A. We also say in this case that A is
a regular matrix.

9 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

Remark 1.6.2
A matrix can be invertible only if it is a square matrix.

Denote GLp (K) the set of all invertible matrices of Mp (K). It is called the General linear group of
degree p.

Example 1.6.3
Ç å
1 2
1. It is easy to verify that if A = then, the inverse of A, denoted A≠1 , is A≠1 =
3 4
Ç å Ç å
≠2 1 1 2
3 1
. Besides, if B = 4
then, the inverse of B, denoted B ≠1 , is B ≠1 =
≠ 3
Ç 22 2
5
å 5
≠ 13 13
15 5
.
26 ≠ 26
Ç å
a b
2. De�ne A the element of M2 (K) by setting: A := . We will show, during recitations,
c d
that A is invertible if and only if ad ≠ bc ”= 0. When this is the case, we have the equality:
Ç å
1 d ≠b
A ≠1
= · .
ad ≠ bc ≠c a

Remark 1.6.4
1. If A is invertible and if G · A = I, then G = GAA≠1 = A≠1 . Likewise, if AD = I, then
D = A≠1 . In particular, the inverse of a matrix, when it exists, is unique.

2. If there are two matrices G and D such that GA = AD = I, then we have G = GAD = D
and A is invertible and its inverse is A≠1 = G = D.

3. We will see later that, if there exists a matrix G such that GA = I, then A is an invertible
matrix and A≠1 = G.

Proposition 1.6.5
Let A and B be two matrices in GLp (K) then:

1. AB is an invertible matrix and its inverse is B ≠1 A≠1 . In other words, the inverse of a product
of matrices is the product of the inverses, but in reverse order.

2. tA is invertible and its inverse is t (A≠1 ).

Proof 1.6.6
1. The fact that AB is invertible comes from the fact that AB · B ≠1 A≠1 = A · A≠1 = Ip .
Theunicity of an inverse, when it exists leaves no other choice but B ≠1 A≠1 for the inverse of
AB.

10 S����� 2023
L����� A������ Lecture Notes C������ 1: Matrices Calculus

2. It is su�cient to show that tA · t A≠1 = Ip . For all (i, j) in J1, pK2 , one can write:
p
ÿ p
ÿ p
ÿ
A · t A≠1
t
i,j
= (tA)i,k (tA≠1 )k,j = ak,i (A≠1 )j,k = (A≠1 )j,k Ak,i = A≠1 · A j,i
k=1 k=1 k=1
= (Ip )i,j .

This latter equality shows that tA · t A≠1 = Ip and achieves the proof.

Example 1.6.7
1. Let A and B be the two matrices de�ned at Example 1.6.3. It is easy to check that:
Ç 18
å Ç 23 9
å
7 5 26 ≠ 26
(A · B) ≠1
= 46
≠1
=
15 5 ≠ 75
52
35
52

while Ç 2 å Ç å Ç å
5 23 9
≠ 13 13 ≠2 1 26 ≠ 26
B ≠1
·A ≠1
= 15 5
· 3 = .
26 ≠ 26 2 ≠ 12 ≠ 75
52
35
52

2. Besides,

Ç å≠1 Ç å
1 3 ≠2 32
(t A)≠1 = =
2 4 1 ≠ 12
&
Ç å Ç å
≠2 1 ≠2 32
t
(A ≠1
)= t
3
= .
2 ≠ 12 1 ≠ 12

11 S����� 2023
C������ 2

Linear Systems

2.1 Linear System as a Matrix Equation

A system with p linear equations with q unknowns is of the form:
8
>
>
> a11 x1 + a12 x2 + · · · + a1q xq = b1
>
< a x + a x + ··· + a x = b
21 1 22 2 2q q 2
(S)
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
>
: a x + a x + ··· + a x = b
p1 1 p2 2 pq q p

where aij and bj are elements of K and the xi are the unknowns that we want to determine. A solution
of the system is a q-tuple denoted (x1 , x2 , · · · , xq ) of Kq for which the p equalities are satis�ed. We
are now interested to the the set of all solutions of the system. Let’s introduce the following notations:
à í à í
x1 b1
x2 b2
A := (ai,j ), X := .. , & B := .. .
. .
xq bp

Using the notations de�ned, it is clear that (S) can be written as

(SÕ ) AX = B.

Matrix A is called matrix of the system and the column matrix B is called second member of the system.
When B = (0) (i.e. bi = 0, for all 1 6 i 6 p), the system is said to be homogeneous. In this case it has
an obvious solution that is the null solution (0, · · · , 0). This latter is also called trivial solution.

De�nition 2.1.1 (Equivalent Systems)

Two linear systems of equations, which have the same number of unknowns are said to be equivalent
if they both have the same set of solutions.

Remark 2.1.2
Two equivalent linear systems have the same number of unknowns but they might have di�erent

12
L����� A������ Lecture Notes C������ 2: Linear Systems

numbers of equations. Hence the two systems:

®
3x ≠ y = 0
(S1 ) : and (S2 ) : 3x ≠ y = 0.
≠15x + 5y = 0

are equivalent.

Proposition 2.1.3
Let A be a p◊q matrix and let G be a p◊p invertible matrix. The systems AX = B and GAX = GB
are equivalent.

Proof 2.1.4
The proof is easy. Indeed, assuming AX = B holds, we just have to multiply, from the left hand
side, both sides of the previous equality by G to get GAX = GB.
Conversely, if GAX = GB holds, we just have to multiply, from the left hand side, both sides of the
previous equality by G≠1 to get AX = B. ⇤

Remark 2.1.5
In the particular case where A is an invertible matrix (i.e. A belongs to in GLp (K)),
then this system is equivalent to the system X = A≠1 B, which has a unique solution
((A≠1 B)1 , (A≠1 B)2 , · · · , (A≠1 B)p ). Such a system is called a Cramer (or regular) system.

2.2 Elementary Operations

This whole paragraph can be skipped at a �rst read, since it is very formal and since it will
be easy to understand once we have performed computations with matrices.
Let’s translate, in term of matrix operations, the method of the Gaussian elimination for solving linear
systems.
In this method we only use three elementary operations, which transform the initial system in an
equivalent system (which therefore has the same solutions as the initial one). These operations are:
1. Multiplication of an equation by a scalar.
More precisely, denote M(⁄, i) the multiplication of the ith equation by the scalar ⁄ (which is assumed
to be di�erent from 0). Note that this means multiply the ith row of the matrix A by the scalar ⁄.
2. Interchange two equations of a system.
More precisely, denote P(i, j) the operation which consists of interchanging the ith and j th equations.
Note that this means interchanging rows i and j of the matrix with each other.

3. Adding an equation to another.

More precisely, denote S(i, j) the operation which consists of adding the j th equation to the ith one.
Note that this means exactly adding the j th row of A to the ith one.
As we will see each of these operations is nothing but multiplying the matrix A of the system (SÕ )
by an invertible matrix, which is said to be associated to the elementary operation. As we will notice
further, the matrix associated to any of the elementary operations described above can be obtained by
applying this elementary operation to the unit matrix.

13 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

1. Multiplication of a row by a scalar.

Let ⁄ be a positive number. Consider the p ◊ p matrix de�ned by:

0 1
1 0 ··· ··· ··· ··· 0 0 1
B
B0 . . . . . . .. C
C 1
B . C B C
B. .. C B1C
B. .. .. C B C
B. . 1 . . C B1C
B C B C
B .. .. .. .. C th B C
M (⁄, i) := B . . ⁄ . . C Ω i rowB1C
B C B C
B. .. .. . C B1C
B .. . 1 . .. C B C
B C B C
B.
.. ..
C @1A
B. . . 0 C
@. A 1
0 ··· ··· ··· ··· 0 1

Let A be a matrix with p rows. We easily see that the matrix M (⁄, i) · A has the same rows as A except
for the ith row, for which all coe�cients has been multiplied by ⁄.
This means that, applying the operation M(⁄, i) to the system AX = B gives us the system

M (⁄, i) · AX = M (⁄, i) · B.

It is also easy to see that the operation M( ⁄1 , i) "cancels" the e�ect of M(⁄, i). The consequence of
this fact is that Å ã
1
M (⁄, i) · M , i = I.
⁄
In other words, for all ⁄ in Rú , M (⁄, i) is invertible and:
Å ã
1
M (⁄, i) = M
≠1
,i .
⁄

2. Interchanging two rows of a Matrix with each other.

Consider the p ◊ p matrix de�ned by:

0 10 1
1 0 ··· 0 ··· 0 ··· 0 0
B0 . . .
B .. .. .. C B .. C
B . . .C CBB .
C
C
B. CB C
B. CB
B. 1 0 · · · 0C B 0 C
C
B CB
B0 · · · 0 ··· 0 1 0 · · · 0C B Ω ith row C
C
B CB C
B CB
B 1 0 CB C
C
B CB C
P (i, j) := B ... .. .. .. .. C B ..
B C
B . . . .C CB . C
B CBB C
B 0 1 CB C
C
B CB C
B0 · · · 0 1 0 ··· 0 · · · 0C BΩ j rowC
C th
B B C
B
B .. C
C B .. C
B 0 1 .CB B . C
C
B. .. .. CB
B. .. C@ 0
C
@. . . . 0A A
0 ··· 0 ··· 0 ··· 0 1 1

Let A be a matrix with p rows. We easily see that the matrix P (i, j) · A has the same rows as A except
for that the rows i and j have been interchanged.

14 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

This means that, applying the operation P(i, j) to the system AX = B gives us the system

P (i, j) · AX = P (i, j) · B.

It is also easy to see that the operation P(j, i) "cancels" the e�ect of P(i, j). The consequence of
this fact is that
P (i, j) · P (j, i) = I.
In other words, for all (i, j) in J1, pK2 , P (i, j) is invertible and:

P (i, j)≠1 = P (j, i).

The P (i, j) matrices are called permutation matrices.

3. Adding a row of a Matrix to another one.

Consider the p ◊ p matrix1 de�ned by:

0 10 1
1 0 ··· 0 ··· 0 ··· 0 0
B0 . . .
B .. .. .. C B .. C
B . . .C CB .
C
B. CBB
C
C
B. CB
B. 0 CB 0 C
C
B CB C
B0 · · · 0 1 0 ··· 0 · · · 0C B Ω ith row C
B CB C
B CB
B 0 CB C
C
B CB C
S(i, j) := B ... .. .. .. .. C B ..
B C
B . . . .CCBB . C
B CB C
B 0 0 CB C
C
B CB
B0 · · · 0 1 0 ··· 0 1 0 · · · 0C B Ω j th rowC C
B CB C
B
B .. C C B .
. C
B 0 0 .CB B . C
C
B. .. .. CB
B. .. C@ 0
C
@. . . . 0A A
0 ··· 0 ··· 0 ··· 0 1 1

Let A be a matrix with p rows. We easily see that the matrix S(i, j) · A is obtained from A by replacing
its j th row by the sum of the ith and j th rows. applying operation S(i, j) to the system AX = B
gives us the system
S(i, j) · AX = S(i, j) · B.
"Canceling" operation S(i, j) can be done by subtracting the ith row of the new matrix (i.e. S(i, j) · A)
to its j th row. More precisely we can do that in three steps.

(i) Changing the sign of the ith row.

(ii) Add the new ith row to the j th row

(iii) Changing again the sign of the ith row.

In other words, for all (i, j) in J1, pK2 , the matrix S(i, j) is invertible and:

S(i, j)≠1 = M (≠1, i) · S(j, i) · M (≠1, i).

4. Generalization of the three elementary operations.

1
We here chose i < j. The reader will write the case where i > j.

15 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

One can generalize the three previous operations. Indeed, for any non zero real ⁄,
Å ã
1
S(⁄, i, j) := M , i · S(j, i) · M (⁄, i).
⁄

corresponds to the operation:“adding to the j th row the ith row multiplied by ⁄”. We denote S(⁄, i, j)
this operation.

Remark 2.2.1
Note that the matrices of these successive operations are written from the right to the left since,
every time, one multiplies from the left hand side.

One easily sees that S(⁄, i, j) is an invertible matrix. Moreover, one has:
Å ã
1
S(⁄, i, j) := M ≠ , i · S(j, i) · M (≠⁄, i) = S(≠⁄, i, j).
≠1
⁄

2.3 Gaussian Elimination & Row Reduced Echelon Matrices

De�nition 2.3.1 (Row Echelon Matrix)

A p ◊ q matrix R is said to be in row echelon form if it has the following “staircase” structure:
0 1
~ ú ··· ú ú ··· ú ú ··· ··· ú ú ú ··· ú
B C
B 0 0 ··· 0 ~ ··· ú ú ··· ··· ú ú ú ··· ú C
B C
B 0 0 ··· 0 0 ··· 0 ~ ··· ··· ú ú ú ··· ú C
B C
B . .. .. . .. .. .. .. .. .. .... .. C
B .. . . .. . . . . . . . . . C
B C
R=B C.
B 0 0 ··· 0 0 ··· 0 0 ··· ··· 0 ~ ú ··· ú C
B C
B C
B 0 0 ··· 0 0 ··· 0 0 ··· ··· 0 0 0 ··· 0 C
B C
B .. .. .. . .. .. . . ..
. .. . .. ..
C
@ . . . .
··· 0 0 0 ··· 0 A
0 0 ··· 0 0 ··· 0 0 ··· ··· 0 0 0 ··· 0

The entries indicated by ~ are the pivots, and must be nonzero. The �rst r rows of R each contain
exactly one pivot, but not all columns are required to include a pivot entry. The entries below the
“staircase”, indicated by the solid line, are all zero, while the non-pivot entries above the staircase,
indicated by stars, can be anything. The last p ≠ r rows are identically zero and do not contain any
pivots. there may, in exceptional situations, be one or more all zero initial columns.

Example 2.3.2
The matrix â ì
4 1 0 ≠4 15 9
0 ≠2 1 6 7 ≠2
R :=
0 0 0 0 ≠3 ≠1
0 0 0 0 0 0
is in row echelon form. It has three pivots that are: 4, ≠2 and ≠3.
Note moreover that the row echelon matrix can have several initial columns consisting of all zeros.

16 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

As the exemple below shows.

á ë
0 0 0 2 5 ≠8 3
S := 0 0 0 0 0 ≠3 1 .
0 0 0 0 0 0 ≠9

This matrix is also in row echelon form. It has three pivots that are: 2, ≠3 and ≠9.

De�nition 2.3.3 (Reduced Row Echelon Matrix)

A p ◊ q matrix R is said to be in reduced row echelon form if it has the “staircase” structure given
at De�nition 2.3.1 and if all the pivots equal 1 and if all the elements above pivots, (in the columns
which contains a pivot), equal 0. In this latter case, we call pivot columns the columns which contain
the pivots.

Example 2.3.4
The matrix 0 1
0 0 1 ú ú ú 0 ú 0 ú ú
B C
B0 0 0 0 0 0 1 ú 0 ú úC
B C
T := B
B0 0 0 0 0 0 0 0 1 ú úCC
B C
@0 0 0 0 0 0 0 0 0 0 0A
0 0 0 0 0 0 0 0 0 0 0
is in reduced row echelon form. Moreover, the pivot columns are here highlighted in bold font.

De�nition 2.3.5 (Rank of a Matrix: A �rst de�nition)

A p ◊ q matrix R is said to be of rank r (where 0 6 r 6 min{p, q}) if it can be obtained, from the
r ◊ r unit matrix, from one hand by adding p ≠ r rows of zeros at the bottom of the matrix and,
from the other hand, by adding between columns j and j + 1 (for j from 0 to r) as many number of
columns of the form 0 10 1
ú 1
B.CB
B .. C B 1
C
C
B CB C
B CB
BúC BΩ j rowC
th
C
B CB C
B0C B 1 C
B CB C
B.CB C
B.C@
@.A 1 A
0 1

where ú denotes any element of K.

Example 2.3.6
The rank of Matrix T , given at Example 2.3.4, is 3.

We hence proved the following result, the proof of which is obvious.

17 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Proposition 2.3.7
The rank of a reduced row echelon matrix is its number of pivots.

Remark 2.3.8
In the particular case where R is reduced row echelon matrix with rank 0, we see that one can only
add columns full of zeros. Thus all the coe�cients of R are zeros. Thus R = (0).

If it is easy to determine the rank of a reduced row echelon matrix. However, one needs a concrete
way to determine the rank of any matrix. This can be done thanks to the following result, the proof of
which gives a concrete way to do so.

Theorem 2.3.9 (Gauss)

Let A be a matrix in Mp,q (K). There exists a unique invertible matrix in Mp (K), denoted G, such
that GA is a reduced row echelon matrix. In other words:

GA = R.

We can now give the de�nitive de�nition of the rank of a Matrix

De�nition 2.3.10 (Rank of a Matrix: Second de�nition)

We de�ne the rank of a p ◊ q matrix as being the rank of the reduced row echelon matrix associated
to A in the Gauss theorem. We denote it r.
In particular, 0 6 r 6 min{p, q}.

Let us now prove Theorem 2.3.9, the proof of which is known as Gaussian Elimination.

Proof 2.3.11
Unicity:
The unicity of such a matrix G, assuming it exists, is a straight consequence of the unicity of the
inverse of a given matrix (see 1.6.4).
Existence of G:
The Matrix G will be built as a product of matrices associated to the elementary operations described
in Section 2.2. In view of Proposition 2.1.3, Matrix G will be invertible since it is a product of invertible
matrices.
We therefore just have to show that, by a succession of elementary operations, one can transform
Matrix A := (ai,j )(i,j)œJ1,pK◊J1,qK into a reduced row echelon matrix.

1. Reduction of the �rst column of A

Assume that the the �rst column of A has at least one non zero coe�cient, say ak,1 . By multiplying
row k by a1k1 we now are in the case where the �rst column of A has one coe�cient, namely ak1 ,
equals to 1 exactly. By interchanging rows 1 and k, one can reduce our reasoning to the case where
this coe�cient (equal to 1) is on the �rst row of A.
If we add successively, for i from 2 to p, the �rst row multiplied by ≠ai1 to the ith row, we get a
matrix the �rst column of which is full of zeros except for the �rst element which equals 1. One can

18 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

summarize this process by writing:

0 1 0 1 0 1 0 1 0 1
a11 · · · a a11 · · · a 1 ···
B C B C B C B C B Õ C
Ba21 C B a C Ba21 C B a C Ba21 C
B Å ãC B C B C
B
B ..
C
C B 1 C
B
B ..
C
C B C B .. C
B . C BM B . B a C B . C
B C B ,i C
C B
C
C B C B C
B≠≠≠≠≠a≠i1≠≠≠æC B P (1, i) C B . C
Ba
B i1 · · ·C
C
B C
B 1 · · ·C
B C B≠≠≠≠≠æC B .. · · ·C
B . C B C B . C B C B C
B . C B .. C B . C B .. C B .
B .
C
@ . A @ . A @ . A B
@ . C
A @ .
C
A
ap1 · · · ap1 ap1 · · · ap1 Õ
ap1 · · ·
0 1 0 1 0 1 0 1
0 1
a 1 ··· a a
1 ···
B C B C B C B C
B
B
a C
C B 0 C B
B a C
C
B
B
a C
C
B
B0
C
C
B C B Õ C B C B C B C
B a C Ba C B a C B a C B0 C
B C B 31 C B C B C B C
B S ≠a21 , 1, 2 C
Õ B . C B S(1) C B S ≠ap1 , 1, p C
Õ B C
B≠≠≠≠≠≠≠≠≠≠æC B .. · · ·C B≠≠≠æ · · ·C B≠≠≠≠≠≠≠≠≠≠æC B0 · · ·C
B C B C B C B C B C
B . C
B .. C B . C B .. C B .. C B
@0
C
B
@ .
C
A @ . A B
@ . C
A
B
@ .
C
A
A
Õ
ap1 · · · 0 ···
ap1 ap1 ap1
0 1
a11 · · ·
B C
Ba21 C
B C
B .. C
B . C
In other words, in order to go from a matrix A, the �rst column of which was B
B C to a matrix
C
a
B i1 · · · C
B . C
B . C
@ . A
ap1 · · ·
0 1
1 ···
B C
B0 · · ·C
B C
B .. C
B. C
the �rst column of which is B
B C we multiplied A, from the left hand side, by the matrix
C
B0 C
B. C
B. C
@. A
0 ···
Å ã
1
G := S(≠ap1 , 1, p) · · · S(≠a21 , 1, 2) · P (1, i) · M
Õ Õ
,i .
ai1
2. Reduction of the other columns of A

In order to prove the theorem we do an induction on p. The �rst part of the demonstration proved
that, by elementary operations on the rows of A, can get a matrix the �rst column of which is of the
form: à í
Ü ê 1
0
.. 0
. or .. .
.
0
0
• If q = 1 (or if p = 1)

19 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

then this ends the proof and the matrix we obtain is in reduced row echelon form. Moreover it has
a rank equal to 0 or 1).

• Assume that If q > 2 and that the theorem is proved up to the order q ≠ 1.

Once we reduced the �rst column (as we did in 1), one has to get the reduced row echelon form of a
matrix, denoted A1 , which is of one of these types
à í
Ü ê 1 ú ··· ú
0
.. 0
. B ..
. B
0
0

where B is a matrix with q ≠ 1 columns. According to the Induction Assumption, one can transform,
using only elementary operations on the rows, the matrix B in a reduced row echelon matrix.
By applying these elementary operations on the matrix A1 (taking into account the shifting between
line numbering of B and A1 (in the second case), we get a matrix A2 which has one of the form
below, but in which B is reduced.
In the �rst case, we are done since by adding a column full of zeros at the left of a reduced row
echelon matrix we still get a reduced row echelon matrix. In the second case, we just have to make
appear zeros on the �rst row of the matrix A2 and only for the pivot columns of matrix B. Denote
j the index of one of these columns. This column is of the form
0 10 1
– 1
B CB C
B0C B 1 C
B CB C
B .. C B 1 C
B.CB C
B CB C
B0C B
B CB 1 C
C
B CB th C.
B 1 C BΩ j rowC
B CB C
B CB
B0C B 0 C
C
B CB
B .. C B .. C
C
@.A@ . A
0 0

We therefore only have to add to the �rst row of A2 , the j th row of A2 multiplied by ≠– in order to
get the desired form for the pivot column. This does not a�ect the columns of A2 which extend the
pivot columns of B which have a di�erent index from j (but modify in general the other columns).
This achieves the proof of the theorem. ⇤

Remark 2.3.12
Let A be in Mp,q (K). In order to determine the matrices G and R := GA, one can use Gaussian
Elimination. In order to avoid having to remember the elementary operations we have done to reach
this goal, we use the fact that G is obtained by performing all these elementary operations on the
unit matrix. Thus we are going to perform the required elementary operations on the matrix A and,
in parallel, on the unit matrix Imin{p,q} . Once the matrix A has been transformed into a reduced row
echelon matrix R, Imin{p,q} will have been transformed into the matrix G.

In the next example, we show how to get the reduced row echelon form of 2 ◊ 3 matrix. In the sequel,
when it comes to get the reduced row echelon form of a given matrix, we will use extensively, the

20 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

following notation: Ri Ω≠ Ri ≠ 2Rj . It means that the new row i is the former row i, from which we
subtracted 2 times the former row j

Example 2.3.13
Ç å
2 ≠2 3
Give the row reduced echelon form of the matrix A := .
4 3 7
We start with the following position:
Ö è
1 0 Ç å Ç å
2 ≠2 3 1 0
Starting positon
4 3 7 0 1
0 1
Ö è
1 0 Ç å Ç å
2 ≠2 3 1 0
Operation S(≠2, 1, 2): i.e. R2 Ω≠ R2 ≠ 2R1
0 7 1 ≠2 1
0 1
Ö è
1 0 Ç å Ç å
1 1 ≠1 3/2 1/2 0
Operation M( 2 , 1) : i.e. R1 Ω≠ R1 /2
0 7 1 ≠2 1
0 1
Ö è
1 0 Ç å Ç å
1 ≠1 3/2 1/2 0
Operation M( 17 , 2) : i.e. R2 Ω≠ R2 /7
0 1 1/7 ≠2/7 1/7
0 1
Ö è
1 0 Ç å Ç å
1 0 23/14 3/14 1/7
Operation S(1, 2, 1) : i.e. R1 Ω≠ R1 + R2
0 1 1/7 ≠2/7 1/7
0 1
Ç å Ç å
1 0 23/14 3/14 1/7
The matrix R := is in reduced row echelon form of A and G := is
0 1 1/7 ≠2/7 1/7
the unique invertible matrix such that GA = R. Here, it is easy to verify:
Ç 3 1
å Ç å Ç 23
å
14 7 2 ≠2 3 1 0 14
G·A= · = = R.
≠ 27 1
7 4 3 7 0 1 1
7

Remark 2.3.14
Since the notation Ri Ω≠ aRi + bRj is a more instinctive way to denote the elementary operations
we are performing, when one tries to determine the reduced row echelon form, we will exclusively
use it from now on. We will therefore only write R1 Ω≠ R1 + R2 instead of Operation S(1, 2, 1) :
i.e. R1 Ω≠ R1 + R2 .

2.4 Application of Gaussian Elimination to Inversion of Matrices

We will focus, in this section, on square matrices.
We saw in the previous section that if the reduced row echelon matrix associated to a square matrix
A by the Gauss Elimination Theorem is the identity matrix then A is an invertible matrix. The Gauss

21 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Elimination then allows not only to establish that a matrix A is invertible, by showing that R = I, but
also to compute its inverse G (since GA = R = I).

Example 2.4.1
Ö è
2 ≠2 3
Let us show that A := 4 1 3 is invertible and let us determine its inverse. Before doing so,
2 0 3
let us remind that R2 Ωæ R3 means that we interchange Row 2 and Row 3.

R2 Ω≠ R2 ≠ 4R1 Ö è Ö è
2 ≠2 3 1 0 0
Starting positon
4 1 3 0 1 0
1
2 0 3 0 0 1
1
Ö è Ö è
R1 Ω≠ R21 1 ≠1 3/2 1/2 0 0
R2 Ω≠ R2 ≠ 4R1 4 1 3 0 1 0
R3 Ω≠ R3 ≠ 7R1 2 0 3 0 0 1
Ö è Ö è
R1 Ω≠ R1 ≠ 2R2 1 ≠1 3/2 1/2 0 0
R2 Ω≠ R2 ≠ 4R1 0 5 ≠3 ≠2 1 0
R3 Ω≠ R3 ≠ 2R1 0 2 0 ≠1 0 1
Ö è Ö è
R2 Ω≠ ≠R2 /3 1 ≠1 3/2 1 0 0
R2 Ωæ R3 0 2 0 ≠1 0 1
R2 Ω≠ R2 ≠ 2R1 0 5 ≠3 ≠2 1 0
Ö è Ö è
R2 Ω≠ ≠R2 /3 1 ≠1 3/2 1 0 0
R2 Ωæ 2 R2
0 1 0 ≠1/2 0 1/2
R2 Ω≠ R2 ≠ 2R1 0 5 ≠3 ≠2 1 0
Ö è Ö è
R1 Ω≠ R1 + R2 1 0 3/2 1/2 0 1/2
R1 Ω≠ R1 ≠ R1 0 1 0 ≠1/2 0 1/2
R3 Ω≠ R3 ≠ 5R2 0 0 ≠3 1/2 1 ≠5/2
Ö è Ö è
R1 Ω≠ R1 + R3 1 0 3/2 1/2 0 1/2
R2 Ω≠ R2 ≠ 2R3 0 1 0 ≠1/2 0 1/2
R3 Ω≠ ≠ 3R3
0 0 1 ≠1/6 ≠1/3 5/6
Ö è Ö è
R1 Ω≠ R1 ≠ 32 R3 1 0 0 1/4 1/2 ≠3/4
R2 Ω≠ R2 ≠ 2R3 0 1 0 ≠1/2 0 1/2
R3 Ω≠ ≠ 3 R3
0 0 1 ≠1/6 ≠1/3 5/6

Since the reduced row echelon matrix is the unit matrix, it appears that A is invertible and that its
inverse is: Ö 1 è
1
4 2 ≠ 34
A≠1 = ≠ 12 0 1
2
.
≠ 16 ≠ 13 5
6

22 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Ö è Ö 1 1
è
2 ≠2 3 4 2 ≠ 34
We can therefore check that 4 1 3 · ≠ 12 0 1
2
= I3 .
2 0 3 ≠ 16 ≠ 13 5
6

We end this section by the following result, which be useful in the next section.

Proposition 2.4.2
Let A be a p ◊ q matrix and F be a p ◊ p invertible matrix. Matrices A and F A have the same rank.

Proof 2.4.3
Let G be an invertible matrix such that R = GA is a reduced row echelon matrix. One can write:

GA = R ≈∆ GF ≠1 · F A = R.

Since GF ≠1 is invertible, this shows that A and F A have the same reduced row echelon matrix, and
thus the same rank. ⇤

2.5 Application of Gaussian Elimination to LU Factorization

We will focus, in this section again, on square matrices.
Introduced by Polish mathematician Tadeusz Banachiewicz in 1938, lower–upper LU decomposition or
factorization factors a matrix as the product of a lower triangular matrix and an upper triangular matrix.
Solving a triangular2 linear system is easy and easier than solving a non triangular linear system. We
therefore understand that it might be of interest to transform the research of the solution of a given
linear system by the research of the solution of triangular systems. This is one of the interests of the
LU Factorization. As we will se at the end of this section, this technique is very useful in numerical
analysis and in linear algebra. Indeed, computers usually solve square systems of linear equations using
LU decomposition, and it is also a key step when inverting a matrix or computing the determinant of
a matrix.
De�nition 2.5.1 (LU Factorization)
Let A be in Mp (K). A presentation A = LU , in which L belongs to Mp (K) and is a lower triangular
and U belongs to Mp (K) and is upper triangular, is called an LU factorization of A.
Ç å
0 1
As we will see later, not all square matrices admit an LU factorization. Hence B := does not
1 0
admit such a decomposition. It is therefore important to be able to know which sets of square matrices
do admit such a decomposition. Let us start by a de�nition.

De�nition 2.5.2 (Regular Gaussian Elimination for Square Matrices)

We say that a Gaussian elimination is regular if the Gaussian elimination algorithm successfully
reduces the given matrix A to a row echelon matrix:

1. only by the elementary operations described at Section 2.2, at the exception of interchang-
ing rows.

2. the pivots obtained via the Gaussian elimination algorithm are all non-zero pivots.

2
i.e. a linear system for which Matrix A is triangular

23 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

By extension one can de�ne regular square matrices.

De�nition 2.5.3 (Regular Square Matrices)

We say that a square matrix is regular if the Gaussian Elimination Algorithm to get the row echelon
form of A successfully reduces to upper triangular form U with all non-zero pivots and without any
interchange of rows.

Example 2.5.4
De�ne the following matrices
Ö è Ö è Ö è
2 ≠2 3 0 2 0 0 6 0
A := 4 1 3 , C := 2 0 0 , & F := 2 1 1 .
2 0 3 0 0 2 0 1 8

It is easy to check that A is regular, while neither C nor F are.

Remark 2.5.5
Careful readers recognized that Matrix A, given in Example 2.5.4 is the same as the one given in
Example 2.4.1. Since, we interchanged row 2 and row 3, in the Gaussian Elimination performed in
Example 2.4.1, it might be surprising to state that A is regular. In fact, one can perform Gaussian

24 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

elimination of A, without interchanging any rows, as we do it right below.

R2 Ω≠ R2 ≠ 4R1 Ö è Ö è
2 ≠2 3 1 0 0
Starting positon
4 1 3 0 1 0
1
2 0 3 0 0 1
1
Ö è Ö è
R1 Ω≠ R21 1 ≠1 3/2 1/2 0 0
R2 Ω≠ R2 ≠ 4R1 4 1 3 0 1 0
R3 Ω≠ R3 ≠ 7R1 2 0 3 0 0 1
Ö è Ö è
R1 Ω≠ R1 ≠ 2R2 1 ≠1 3/2 1/2 0 0
R2 Ω≠ R2 ≠ 4R1 0 5 ≠3 ≠2 1 0
R3 Ω≠ R3 ≠ 2R1 0 2 0 ≠1 0 1
Ö è Ö è
R2 Ω≠ ≠R2 /3 1 ≠1 3/2 1/2 0 0
R2 Ω≠ R2 /5 0 1 ≠3/5 ≠2/5 1/5 0
R2 Ω≠ R2 ≠ 2R1 0 2 0 ≠1 0 1
Ö è Ö è
R1 Ω≠ R1 + R2 1 0 9/10 1/10 1/5 0
R1 Ω≠ R1 ≠ R1 0 1 ≠3/5 ≠2/5 1/5 0
R3 Ω≠ R3 ≠ 2R2 0 0 6/5 ≠1/5 ≠2/5 1
Ö è Ö è
R1 Ω≠ R1 + R3 1 0 9/10 1/10 1/5 0
R2 Ω≠ R2 ≠ 2R3 0 1 ≠3/5 ≠2/5 1/5 0
5
R3 Ω≠ 6 R3 0 0 1 ≠1/6 ≠1/3 5/6
9
Ö è Ö è
R1 Ω≠ R1 ≠ 10 R3 1 0 0 1/4 1/2 ≠3/4
3
R2 Ω≠ R2 ≠ 5 R3 0 1 0 ≠1/2 0 1/2
R3 Ω≠ ≠ 3R3
0 0 1 ≠1/6 ≠1/3 5/6

The reason why we performed Gaussian elimination the way we did it, in Example 2.4.1, was to
9
simplify computations (and not get a ≠ 10 or 35 for example).

LU Factorization of Regular Matrices

Regular matrices have a LU factorization. Moreover one can be accurate on the value of the pivots of
both L and U . This is the object of the next result, the proof of which is admitted here but which can
be found in [HJ13, Theorem 3.5.2 & Corollary 3.5.5, p.160 to 162].

Theorem 2.5.6 (Regular LU Factorization)

A matrix A is regular if and only if it can be factored

A = LU (5.1)

where L is a lower unitriangular matrix, having all 1’s on the diagonal, and U is upper triangular with
non zero diagonal entries, which are the pivots of A. The non zero o�-diagonal entries lij for i > j
appearing in L prescribe the elementary row operations that bring A into upper triangular form;

25 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

namely, one subtracts li,j times row j from row i at the appropriate step of the Gaussian Elimination
process.

Remark 2.5.7
In order to avoid any confusion between LU factorization (given at De�nition 2.5.1, and where we
do not have any information about the diagonal entries of U ) and the LU factorization described
right above (where we know that matrix U has non zero diagonal entries), we will call this latter
regular LU factorization.

In practice, to �nd the regular LU factorization of a square matrix A, one applies the regular Gaus-
sian Elimination algorithm to reduce A to its upper triangular form U . The entries of L can be �lled in
during the course of the calculation with the negatives of the multiples used in the elementary row op-
erations. If the algorithm fails to be completed, which happens whenever zero appears in any diagonal
pivot position or if we have to interchange rows to get the row echelon form of A, then the original
matrix does not have an regular LU factorization. Let’s see on an example how the LU factorization
can be performed.

Example 2.5.8
Let’s compute the LU factorization of the matrix
Ö è
1 2 1
A := 3 10 3 .
≠2 ≠8 5
At this stage we do not even know if such a factorization does exist for A. Only the Gaussian Elimi-
nation algorithm will tell us if it does. Thus, applying the Gaussian Elimination algorithm, we begin
by de�ning A1 to be A and L1 to be I3 . We hence start with:
R2 Ω≠ R2 ≠ 2R1 Ö è Ö è
1 2 1 1 0 0
R2 Ω≠ R2 ≠ 4R1
3 10 3 := A1 0 1 0 := L1
1
≠2 ≠8 5 0 0 1
1
Ö è Ö è
R1 Ω≠ R1 ≠ 2R2 1 2 1 1 0 0
R2 Ω≠ R2 ≠ 3R1 0 4 0 := A2 3 1 0 := L2
R3 Ω≠ R3 + 2R1 0 ≠4 7 ≠2 0 1
Ö è Ö è
R1 Ω≠ R1 ≠ 2R2 1 2 1 1 0 0
R2 Ω≠ R2 ≠ 2R1 0 4 0 := A3 3 1 0 := L3
R3 Ω≠ R3 + 1R2 0 0 7 ≠2 ≠1 1
Note that A3 is an upper triangular matrix while L3 is a lower triangular matrix. Thus one de�nes
U to be A3 and L to be L3 . In other words, de�ne matrices L and U by setting:
Ö è Ö è
1 0 0 1 2 1
L := 3 1 0 & U := 0 4 0 .
≠2 ≠1 1 0 0 7
Note that the diagonal entries of U are the pivots of A. Since they are all non zero here, it is clear, in
view of Theorem 2.5.6, that A is regular. Therefore the LU factorization we just gave is the regular
LU factorization of A.

26 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Note that L has always all its diagonal elements equal to 1 while its entries lying below the main
diagonal are the negatives of the multiples we used during the elimination procedure. Moreover, it
is easy to check that:
Ö è Ö è Ö è
1 0 0 1 2 1 1 2 1
L·U = 3 1 0 · 0 4 0 = 3 10 3 = A.
≠2 ≠1 1 0 0 7 ≠2 ≠8 5

Remark 2.5.9
By trying Gaussian elimination on matrices B and C given in Example 2.5.4 we immediately see that
the algorithm requires an interchange of rows. Thus none of these two matrices are regular. As a
consequence none of them admit a regular LU factorization.

As the next subsection will show, that it is possible, if we get the rid of the condition concerning the
interchanging of rows, to extend the regular LU factorization to any invertible matrix (and in fact to a
bigger set of matrices). The price to pay will be that we will not have A = LU anymore but P A = LU ,
for a certain invertible matrix P .

P A = LU Factorization

Before starting this section, we must have in mind, what the permutation matrices are. We de�ned
them at page 14. Hence, it is easy to see that, in M3 (R), there are 3! = 6 permutations matrices that
are:

Ö è Ö è Ö è
0 1 0 0 0 1 1 0 0
P (1, 2) = 1 0 0 P (1, 3) = 0 1 0 P (2, 3) = 0 0 1
0 0 1 1 0 0 0 1 0
Ö è Ö è Ö è
0 1 0 0 0 1 1 0 0
P (2, 1, 3) = 0 0 1 P (3, 1, 2) = 1 0 0 PId = 0 1 0 .
1 0 0 0 1 0 0 0 1

Let us now de�ne Nonsingular Matrix.

De�nition 2.5.10 (Nonsingular Matrix)

We say that a square matrix A is nonsingular if the Gaussian Elimination Algorithm successfully
reduces it to an upper triangular matrix U with all non-zero pivots (on the diagonal then) only by
the elementary rows operations described at Section 2.2.

Remark 2.5.11
As we will see in Theorem 2.5.16, invertible is a synonym of nonsingular, for square matrices.
A matrix which is not nonsingular is said to be singular.

We can now state the most general result, the proof of which can be found in [HJ13, Theorem 3.5.7].

27 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Theorem 2.5.12 (P A = LU Factorization)

For each nonsingular matrix A in Mp (K) there is a permutation matrix P in GLp (K), a unit lower
triangular L in Mp (K), and an upper triangular U in Mp (K) such that P A = LU . We call per-
muted LU factorization such a decomposition.

Remark 2.5.13
A unit lower triangular matrix is just a lower triangular matrix the diagonal entries of which are all
equal to 1.

De�nition 2.5.14 (Nonsingular P A = LU Factorization)

For any non singular matrix A, we call nonsingular P A = LU Factorization or nonsingular
permuted LU Factorization the P A = LU Factorization granted by Theorem 2.5.12.

Remark 2.5.15
The main information added when one goes from P A = LU Factorization to nonsingular P A =
LU Factorization is that, in this latter case all the diagonal entries of U are non zero.

We have the following result.

Theorem 2.5.16 (Nonsingular P A = LU Factorization)

Let A be a matrix in Mp (K). The following conditions are equivalent:

1. A is nonsingular.

2. A is invertible.

3. A has p nonzero pivots.

4. A admits a nonsingular permuted LU factorization: P A = LU .

Proof 2.5.17
1 =∆ 3: by very de�nition of non singular matrices.
3 =∆ 1: is a straight consequence of Gaussian elimination.
3 =∆ 2: Since A has p non zero pivots, the Matrix R, provided by the Gaussian elimination of A,
must equals Ip . Moreover, since the equality GA = R holds, this proves that A is invertible and that
A≠1 = G.
2 =∆ 3: Let us prove this result by using the contrapositive. In other words let us prove that
non 3 =∆ non 2.
Denote G and R the matrices provided by the Gaussian elimination of A. We hence have:

GA = R (5.2)
If A does not have p nonzero pivots, then R can not be Ip . In particular R is not invertible.
Let’s assume that A is invertible. In this case GA itself is invertible, according to Proposition 1.6.5.
Thus, one can write
(5.2) ≈∆ (GA)≠1 · GA = (GA)≠1 R ≈∆ Ip = (GA)≠1 R.
In other words, R is invertible. This contradicts the fact that R is not invertible. Thus our assumption
that A is invretible does not hold.

28 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

1 =∆ 4: this is Theorem 2.5.16.

4 =∆ 1: If A admits a nonsingular permuted LU factorization, then A is non singular. ⇤

A practical method to construct a permuted LU factorization of a given matrix A, whether it is non-

singular or not, of Mp (K) would proceed as follows. First set up P = L = Ip . The matrix P will keep
track of the permutations performed during the Gaussian Elimination process, while the entries of L
below the diagonal are gradually replaced by the negatives of the multiples used in the corresponding
elementary row operations (except the interchanging of two rows). Each time two rows of A are inter-
changed, the same two rows of P will be interchanged as well. Moreover, any pair of entries that both
lie below in these same two rows of L must also be interchanged, while entries lying on and above
its diagonal need to stay in their place. At a successful conclusion to the procedure, A will have been
converted into the upper triangular matrix U , while L and P will assume their �nal form. Here is an
illustrative example.

Example 2.5.18
Let us give a permuted LU factorization of the matrix
à í
1 2 ≠3 4
≠5 ≠10 9 ≠36
A := .
≠2 ≠6 5 ≠5
3 0 ≠9 27
We hence start with:
à í à í à í
1 2 ≠3 4 1 0 0 0 1 0 0 0
≠5 ≠10 9 ≠36 0 1 0 0 0 1 0 0
:= A1 := L1 := P1
≠2 ≠6 5 ≠5 0 0 1 0 0 0 1 0
3 0 ≠9 27 0 0 0 1 0 0 0 1
To begin the procedure, we eliminate the entries below the �rst pivot.
à í à í à í
R2 Ω≠R2 +5R1 1 2 ≠3 4 1 0 0 0 1 0 0 0
R2 Ω≠R2 +5R1 0 0 ≠6 ≠16 ≠5 1 0 0 0 1 0 0
:=A2 :=L2 :=P2 .
R3 Ω≠R3 +2R1 0 ≠2 ≠1 3 ≠2 0 1 0 0 0 1 0
R4 Ω≠R4 ≠3R1 0 ≠6 0 15 3 0 0 1 0 0 0 1
Since the (2, 2) entry of A2 is zero, we interchange rows 2 and 3 which leads us to:
à í à í à í
1 2 ≠3 4 1 0 0 0 1 0 0 0
R2 Ω≠R2 ≠2R1
0 ≠2 ≠1 3 ≠2 1 0 0 0 0 1 0
R2 ΩæR3 :=A3 :=L3 :=P3 .
0 0 ≠6 ≠16 ≠5 0 1 0 0 1 0 0
R4 Ω≠R2 +1R1
0 ≠6 0 15 3 0 0 1 0 0 0 1
We interchanged the same two rows of P , while in L we only interchanged the already computed
entries in tis second and third rows that lie in its �rst column below the diagonal. We then eliminate
the nonzero entry lying below the (2, 2) pivot, leading to:
à í à í à í
R2 Ω≠R2 ≠2R1 1 2 ≠3 4 1 0 0 0 1 0 0 0
R2 Ω≠R3 0 ≠2 ≠1 3 ≠2 1 0 0 0 0 1 0
:=A4 :=L4 :=P4 .
R3 Ω≠R2 0 0 ≠6 ≠16 ≠5 0 1 0 0 1 0 0
R4 Ω≠R4 ≠3R2 0 0 3 6 3 3 0 1 0 0 0 1

29 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

At this stage, another row interchange will simplify the computations:

à í à í à í
1 2 ≠3 4 1 0 0 0 1 0 0 0
R2 Ω≠R2 ≠2R1
0 ≠2 ≠1 3 ≠2 1 0 0 0 0 1 0
R2 Ω≠R3 :=A5 :=L5 :=P5 .
0 0 3 6 3 3 1 0 0 0 0 1
R3 ΩæR4
0 0 ≠6 ≠16 ≠5 0 0 1 0 1 0 0

We now just have to make the matrix A5 under a triangular form:

à í à í à í
1 2 ≠3 4 1 0 0 0 1 0 0 0
R2 Ω≠R2 ≠2R1
0 ≠2 ≠1 3 ≠2 1 0 0 0 0 1 0
R2 Ω≠R3 :=A6 :=L6 :=P6 .
0 0 3 6 3 3 1 0 0 0 0 1
R2 Ω≠R2 ≠2R1
0 0 0 ≠4 ≠5 0 ≠2 1 0 1 0 0

We thus de�ne

U := A6 , L := L6 , & P := P6 .

In other words, we have:

à í à í à í
1 2 ≠3 4 1 0 0 0 1 0 0 0
0 ≠2 ≠1 3 ≠2 1 0 0 0 0 1 0
U := , L := & P := .
0 0 3 6 3 3 1 0 0 0 0 1
0 0 0 ≠4 ≠5 0 ≠2 1 0 1 0 0

U having all non zero pivots, it is, as well as A, non singular. Besides, one can verify that:
à í à í
1 0 0 0 1 2 ≠3 4
0 0 1 0 ≠5 ≠10 9 ≠36
PA = ·
0 0 0 1 ≠2 ≠6 5 ≠5
0 1 0 0 3 0 ≠9 27
à í à í à í
1 2 ≠3 4 1 0 0 0 1 2 ≠3 4
≠2 ≠6 5 ≠5 ≠2 1 0 0 0 ≠2 ≠1 3
= = ·
3 0 ≠9 27 3 3 1 0 0 0 3 6
≠5 ≠10 9 ≠36 ≠5 0 ≠2 1 0 0 0 ≠4
= LU. (5.3)

Remark 2.5.19
1. In order not to overload notations one usually does not specify the nature of the LU factor-
ization (i.e. regular or nonsingular nor if it is or not a permuted one) since it is clear from the
context.

2. Regular (if it exists) or permuted LU factorization is not, in general, unique.

3. Note that when one deals with a non invertible matrix A, a “kind” of permuted LU factorization
exists but there will be some zeros on the diagonal of U (as shows Example 2.6.15 below).

30 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

4. The advantages of such a factorization will be given in Remark 2.6.13.

For more details about whet happens in this case, one can refer to [HJ13, Theorem 3.5.7 p.163]

Readers interested in more details about LU factorization (in particular, when A is singular) should
refer to [HJ13, p.160-163], and in particular to [HJ13, Theorem 3.5.7 p.163]. The �gure below gives a
summary of how the regular, nonsingular and singular matrices are linked to each others.

Let A be in Mp (R), where p œ Nú .

If rk (A) < p If rk (A) = p

then A is singular then A is nonsingular

A is regular A is not regular

See Point 3 of Remark 2.5.19
then A has an then A doesn’t have
LU factorization an LU factorization

(but has a P LU factorization)

The set of square matrices Mp (K)

2.6 Application of Gaussian Elimination to solving Linear System

Let us �rst state the general results that apply to all rectangular linear systems.

Results for general linear systems

Proposition 2.6.1
A system of p linear equations with q unknowns has either 0, or 1 or in�nitely many solutions. If
q > p, it has either 0, or in�nitely many solutions.

Proof 2.6.2
The linear system we are studying has p equations and q unknowns and can be written, under the
matrix form AX = B. According to Gauss theorem, there exists a unique invertible matrix G such
that R = GA is a reduced row echelon matrix. We therefore just have to solve the system RX = GB,
which is equivalent to the initial system. Denote r the rank of the reduced row echelon matrix R.
Since the p ≠ r last rows of matrix R equal zero, the coe�cients of the vector RX all equal 0, for
r < i 6 p. Two cases are therefore possible.
- Coe�cients (GB)i of he column vector GB are not all equal to 0, for r < i 6 p. There is an
equation of the system which has the form

0x1 + · · · + 0xq = –

31 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

with – ”= 0. In this case, the system has no solution.

- Coe�cients (GB)i of he column vector GB are all equal to 0, for r < i 6 p. Thus the p ≠ r
last rows of the system have the form

0x1 + · · · + 0xq = 0.

By cancelling them out, we get a system (of r equations with q unknowns), which is clearly
equivalent to the initial system. Denote j1 , . . . , jr the columns indices of R that contain the
pivots of R. By sending the unknowns that correspond to the columns of R that do not contain
the pivots of R, in the second member of the equation, the ith equation of the system becomes:
ÿ
xji = (GB)i ≠ Rij xj .
16j6q
j”={j1 ,...,jr }

The system therefore gives the r unknowns xj1 , . . . , xjr when one knows the q ≠ r others.

• If r = q, the unknowns are all determined and the system has only one solution, that is
((GB)1 , . . . , (GB)q ).

• If r < q, one can arbitrary choose q≠r unknowns and the system has in�nitely many solutions.

• Particular case p < q: Since r 6 p, one can not have r = q and the system can therefore not
have only one solution.

De�nition 2.6.3 (Rank of a Linear System)

We de�ne the rank of a linear system AX = B, as being the rank of the matrix A.

Corollary 2.6.4
An homogeneous linear system (i.e. with a second member equal to (0)) with p equations and q
unknowns has a non zero solution if and only if the rank of its matrix is smaller than q. This is in
particular the case if p < q.

Proof 2.6.5
The system AX = 0Rq has at least the null solution, i.e. 0Rq . It will have a non zero solution if and
only if it has in�nitely many of them. According to the proof of Proposition 2.6.1, it happens if and
only if r < q. ⇤

Results About Square Linear Systems

The results stated in the previous section remain valid. However, one can be more accurate when it
comes to square linear systems.
From now on, de�ne Let A be a matrix of Mp (K) and B be a column matrix with p rows. Denote

32 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems
à í
x1
x1
X := .. and de�ne the following system:
.
xp

(S) AX = B.
The next result gives the methodology to solve such a linear system if A is invertible, using the inverse
of A.
Theorem 2.6.6 (Characterization of Invertible Matrices)
Considering the System (S) described above, the following conditions are equivalent:

1. A is an invertible matrix.

2. The rank of A equals p (in other words the reduced matrix, associated to A has no null row).

3. The system AX = 0Rp has only one solution which is 0Rp .

4. For all column matrix B, the system AX = B has at least one solution.

5. For all column matrix B, the system AX = B has a unique solution.

Moreover, when one of these conditions holds, then the reduced row echelon matrix, associated to
A, is Ip and the unique solution of the system AX = B is ((A≠1 B)1 , (A≠1 B)2 , · · · , (A≠1 B)p ).

Proof 2.6.7
1 =∆ 5 is a straight consequence of Proposition 2.4.2.
Proof 2.5.17 already established the equivalence 1 ≈∆ 2.
5 =∆ 4 and 5 =∆ 3 are obvious.
It remains to prove 4 =∆ 2 and 3 =∆ 2. We will use the contrapositive and show that non 2 implies
both non 3 and non 4.
Denote G and R the matrices associated to A by Gauss theorem. We hence have GA = R, and R is
a reduced row echelon matrix.
non 2 =∆ non 4:
non 2 means that the rank of A is not equal to p. It is therefore smaller than p. Hence the coe�cient
on the last row of R are all equal to 0. Let B be a column matrix such that the coe�cient, on its last
row, equals 1. System AX = G≠1 B is equivalent to the system RX = B. Since the last equation of
this latter system is:
0x1 + · · · + 0xp = 1,
it has no solution. Hence Condition 4 is false. This proves non 2 =∆ non 4.
non 2 =∆ non 3:
Homogeneous system AX = (0) is equivalent to the system RX = (0). In this latter, one can
suppress the last equation, which is of the form:

0x1 + · · · + 0xp = 0.

We get an equivalent system, which has p≠1 equations and p unknowns. Since it has more unknowns
than equations, Proposition 2.6.1 tells us that it can not have a unique solution. In other words 3 is
not true. This proves non 2 =∆ non 3 and achieves the proof. ⇤

33 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Corollary 2.6.8
Let A be a matrix in Mp (K). The following conditions are equivalent:

1. A is an invertible matrix.

2. There exists G in Mp (K) such that GA = Ip .

3. There exists D in Mp (K) such that AD = Ip .

Proof 2.6.9
1 =∆ 2 and 1 =∆ 3 are obvious.
2 =∆ 1:
Assume that 2 holds. Let X be a column matrix such that AX = 0Rp . By multiplying, from the left
hand side by G, we get: GAX = 0Rp i.e. X = 0Rp . This shows that 0Rp is the only solution of
the system. Condition 3 of Theorem 2.6.6 therefore holds, and thus, establishes that Condition 1 of
Theorem 2.6.6 holds as well.
3 =∆ 1:
Assume that 3 holds. Let B be a column. De�ne X := DB. By multiplying, from the left hand side
by A, we get: AX = ADB = B. Hence Condition 4 of Theorem 2.6.6 holds, and thus, establishes
that Condition 1 of Theorem 2.6.6 holds as well. ⇤

The next result will give a justi�cation to the methodology for solving linear systems, when A is
invertible, using the P A = LU factorization.

Theorem 2.6.10 (Solving linear system with P A = LU factorization)

Let us consider the System (S) described above. Assuming that A is an invertible matrix, (S) admits
a unique solution, that can be found using P A = LU factorization.

Proof 2.6.11
The proof 2.6.7 shows that there is a unique solution to such a system. The fact that one can use
permuted LU -factorization to solve the system becomes clear once one reads the following method-
ology. ⇤

Methodology for solving square linear system using P A = LU factorization

Once the permuted P A = LU factorization is established, the solution to the original system (S),
given at page 33, is obtained by applying the forward and back Substitution algorithm presented above.
Explicitly, we �rst multiply the system AX = B by P , from the left hand side, leading to:

P AX = P B =: B Õ . (6.4)

Note that B Õ has been obtained by permuting the entries of B in the same fashion as the rows of A.
Moreover (6.4) can be rewritten under the form:

LU X = B Õ . (6.5)

We then solve the two triangular systems. The �rst one is

LC = B Õ , (6.6)

34 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

where C is the unknown vector in (6.6), and is solved by Forward Substitution. The second triangular
system to solve, by Backward Substitution this time, is

UX = C (6.7)
in the unknown X. Let’s apply this methodology in the following example.

Example 2.6.12
Let us solve the following linear system:

(T) AX = B,

where
à í à í
1 2 ≠3 4 1
≠5 ≠10 9 ≠36 ≠1
A := and B := .
≠2 ≠6 5 ≠5 3
3 0 ≠9 27 0

1. Using the inverse of A.

The Gauss elimination algorithm shows that A is invertible and that
à í
≠ 231
4
15
4 ≠ 51
2
53
6
69 5 15 17
8 ≠ 8 4 ≠ 12
A≠1 = .
≠ 192
1
2 ≠4 4
3
13
4 ≠ 14 3
2 ≠ 12

According to the conclusion of Theorem 2.6.6, we therefore know that X = A≠1 B. Thus
0 1
(A≠1 B)1 à í à í à í
231 15 51 53
B
B
C
C ≠ 4 4 ≠ 2 6 1 ≠138
B (A≠1 B)2 C 69
≠ 58 15
≠ 17 ≠1 41
B C 8 4 12 2
X=B C= 19 1 4
· = (6.8)
B (A≠1 B) C
B 3 C
≠ 2 2 ≠4 3 3 ≠22
@ A 13
≠4 1 3
≠ 12 0 8
4 2
(A B)4
≠1

De�ne Sol(T) the set of solutions of System (T). We just proved that:
¶Ä ä©
Sol(T) = 41 ,
t
≠138 2 ≠22 8

where t is the transpose symbol.

2. Using the P A = LU factorization.

The P A = LU factorization of A has been computed at (5.3) (on page 30) and is therefore not recalled
here. According to the methodology given right after Theorem 2.6.10, de�ne
à í à í
1 0 0 0 1
0 0 1 0 3
B Õ := ·B = .
0 0 0 1 0
0 1 0 0 ≠1

35 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems
0 1
c1
B C
B C
B c2 C
B C
Denote C := B C. We know that we �rst need to solve, by forward substitution, the system
B c C
B 3 C
@ A
c4

LC = B Õ , (6.9)

in C. Since (6.9) reads

0 1
à í c1 à í
1 0 0 0 B
B
C
C 1
≠2 1 0 0 B c2 C 3
B C
·B C= , (6.10)
3 3 1 0 B c C
B 3 C 0
≠5 0 ≠2 1 @ A ≠1
c4
0 1
1
B C
B C
B 5 C
B C
we easily get the unique solution of (6.9), which is: C = B C. One then has to solve, by
B ≠18 C
B C
@ A
≠32
backward substitution this time, the following system

U X = C, (6.11)

in the unknown X. Since this latter system reads

0 1 0 1
à í x1 1
1 2 ≠3 4 B
B
C B
C B
C
C
0 ≠2 ≠1 3 B x2 C B 5 C
B C B C
·B C=B C (6.12)
0 0 3 6 B x C B ≠18 C
B 3 C B C
0 0 0 ≠4 @ A @ A
x4 ≠32

We easily get the solution, which is t ( ≠138 41

2
≠22 8 ). Thus, both methods (that are inversion of
matrix and LU factorization) provided us with the same solution.

Remark 2.6.13
When it comes to solving a square linear systems AX = B for several di�erent matrices B, the per-
muted LU factorization can be repeatedly applied to solve the equation multiple times for di�erent
B. In this case it is faster (and more convenient) to do a permuted LU decomposition of the matrix
A once and then solve the triangular matrices system for the di�erent B, rather than using Gaussian
elimination. Indeed, inverse matrices are very convenient in analytical manipulations, because they
allow you to move matrices from one side to the other of equations easily. However, inverse matrices
are almost never computed in “serious” numerical calculations. Whenever you see A≠1 B, when you

36 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

go to implement it on a computer you should read A≠1 B as “solve AX = B by some method”: e.g.
solve it by �rst computing the LU factorization of A and then using it to solve AX = B.
One reason that you don’t usually compute inverse matrices is that it is wasteful: once you have
P A = LU , you can solve AX = B directly without bothering �nding A≠1 , and computing A≠1
requires much more work if you only have to solve a few right-hand sides. Another reason is that for
many special matrices, there are ways to solve AX = B much more quickly than you can �nd A≠1 .
For example, many large matrices in practice are sparse (mostly zero), and often for sparse matrices
you can arrange for L and U to be sparse too. Sparse matrices are much more e�cient to work with
than general “dense” matrices because you don’t have to multiply (or even store) the zeros. Even if
A is sparse, however, A≠1 is usually non-sparse, so you lose the special e�ciency of sparsity if you
compute the inverse matrix.
The matrices L and U could be thought to have “encoded” the Gaussian elimination process.
Moreover, The cost of solving a system of linear equations is approximately 23 p3 �oating-point op-
erations if the matrix A belngs to Mp (R). This makes it twice as fast as algorithms based on QR
decomposition, which costs about 43 p3 �oating-point operations when Householder re�ections are
used. For this reason, LU decomposition is usually preferred. The readers interested in these nu-
merical details may refer to [Str16, p.101-102].

The next theorem summarizes the ways of solving a square linear system. The proof is obvious, in
view of Theorem 2.6.6 and Theorem 2.6.10.

Theorem 2.6.14 (Solving Square Linear Systems)

Let A be a matrix in Mp (K) and B be a column matrix with p rows. Let us consider the following
linear system

(S) AX = B,

where we denoted X := t (x1 , x2 , · · · , xp ) the unknown vector. Depending on the rank of the matrix
A, there are di�erent possible issues when one wants to solve (S).

1. 1st possibility: rk(A) = p

In this case A is invertible.

• One can solve (S) by invoquing Point 5. of Theorem 2.6.6. The unique solution of the system
AX = B is ((A≠1 B)1 , (A≠1 B)2 , · · · , (A≠1 B)p ). One just have to compute the inverse of A.

• One can also solve (S) using the P A = LU method (see Theorem 2.6.10 and the methodology
given right below it).

2. 2st possibility: rk(A) < p

In this case A is not invertible.

One can thus use either Gaussian elimination or the permuted LU factorization to determine whether
the system has or not a solution. However, the permuted LU factorization is, in general, faster and
less expensive (in terms of computations) than Gaussian elimination. Note that both these methods
still allows one to described the set of solutions precisely.

Let us �nish this chapter with an example of linear system, the solution of which is not unique.

37 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Example 2.6.15
We want to solve the following linear system.
8
< x≠z =1
>
(S ) : 2y + 3z = 0 .
>
:
≠2x + 2z = ≠2
To this end, de�ne
á ë á ë
Ö è 1 x
1 0 ≠1
A := 0 2 3 B := 0 , & X := y .
≠2 0 2
≠2 z
It is clear that (S ) is equivalent to AX = B. Since the last row of A is proportional to the �rst one,
it is clear that A is not invertible (since its rank is smaller than 3).
1. Using the P A = LU factorization.
One could either use Gaussian elimination or the permuted LU factorization of A to solve (S ). Let
use �rst the permuted LU factorization of A. We hence start with:
Ö è Ö è Ö è
1 0 ≠1 1 0 0 1 0 0
0 2 3 := A1 0 1 0 := L1 0 1 0 := P1
≠2 0 2 0 0 1 0 0 1
The second and last step is therefore:
Ö è Ö è Ö è
1 0 ≠1 1 0 0 1 0 0
0 2 3 := A2 0 1 0 := L2 0 1 0 := P2
0 0 0 ≠2 0 1 0 0 1
We thus de�ne
U := A2 , L := L2 , & P := P2 .
We then have to solve, �rst:
á ë
Ö èÖ è 1
1 0 0 c1
LC = P B i.e. 0 1 0 c2 = 0
≠2 0 1 c3
≠2
á ë
1
The solution of such a system is, obviously, C = 0 . We then have to solve:

0
á ë
Ö èÖ è 1
1 0 ≠1 x1
UX = C i.e. 0 2 3 x2 = 0
0 0 0 x3
0

38 S����� 2023
L����� A������ Lecture Notes C������ 2: Linear Systems

Such a system also reads

®
x1 ≠ x3 = 1
(S Õ )
2x2 + 3x3 = 0
(S Õ ) is equivalent to
®
x1 = 1 + x3
x2 = ≠3x
2
3

We see easily see that (S Õ ) (and thus (S )) has in�nitely many solutions. More precisely, if S ol(S )
denotes the set of solutions of (S ), one can write:
⁄
S ol(S ) = {(1 + –, ≠3–
2 , –), – œ R}.

2. Using Gaussian Elimination.

We just need to get the reduced row echelon form of A. A few computations provides us with:
Ö è Ö è
1 0 ≠1 1 0 0
R := 0 2 3 & G := 0 1/2 0
0 0 0 ≠2 0 1
Since we know, by de�nition of reduced row echelon form, that GA = R, one can write:

(S ) ≈∆ AX = B ≈∆ GAX = GB ≈∆ RX = GB
á ë á ë
Ö è x Ö è 1
1 0 ≠1 1 0 0
≈∆ 0 2 3 y = 0 1/2 0 0
0 0 0 ≠1 0 1
z ≠2
Such a system also reads:
®
x≠z =1
(S ÕÕ )
2y + 3z = 0
It is clear that (S ÕÕ ) is nothing but (S Õ ), with (x, y, z) instead of (x1 , x2 , x3 ). The set of solution
of (S ÕÕ ) is therefore S ol(S ) = {(1 + –, ≠3– 2 , –), – œ R}. We recovered the result from the P LU
factorization.

39 S����� 2023
Part II

40
C������ 3

Vector Spaces

3.1 Vector Space

In the whole chapter K denotes R or C.

De�nition & Properties

De�nition 3.1.1 (Vector Space)

Let E be a set endowed with two laws, denoted + and · such that:

K◊E æ E
(–, x) ‘æ – · x

(E, +, ·) is said to be a Vector Space (or a Linear Space) on K or a K-Vector Space (or a K-Linear
Space) if:

• (E, +) is a commutative (or abelian) group i.e. :

1. ’(x, y) œ E 2 , x+y =y+x

2. ’(x, y, z) œ E 3 , (x + y) + z = x + (y + z)
3. ÷ 0E œ E s.t. ’x œ E, x + 0E = x, 0E is called the identity element of (E, +).
4. ’x œ E, ÷ ≠ x œ E x + (≠x) = 0E .

• ’(x, y) œ E 2 and (–, —) œ K2 , we have:

5. – · (x + y) = – · x + – · y
6. (– + —) · x = – · x + — · x
7. 1 · x = x
8. – · (— · x) = (–—) · x.

We call vectors the elements of E and scalars the elements of K.

For sake of notational simplicity one will write x ≠ y instead of x + (≠y).

41
L����� A������ Lecture Notes C������ 3: Vector Spaces

Example 3.1.2 (A �rst vector space)

Let us prove that (R2 , +, ·), where + denotes the addition of two vectors of R2 and · denotes the
product of a vector of R2 by a scalara , is a vector space.
Ç å Ç å
x1 y1
Let x := and y := be two vectors of R2 and let ⁄ be a real number. Since one can
x2 y2
write:
Ç å Ç å Ç å Ç å Ç å Ç å
x1 y1 x1 + y1 y1 + x1 y1 x1
1. x+y = + = = = + =y+x
x2 y2 x2 + y2 y2 + x2 y2 x2
ÇÇ å Ç åå Ç å Ç å
x1 y1 z1 x1 + y1 + z1
2. (x + y) + z = + + =
x2 y2 z2 x2 + y2 + z2
Ç å Ç å Ç å ÇÇ å Ç åå
x1 y 1 + z1 x1 y1 z1
= + = + + = x + (y + z)
x2 y 2 + z2 x2 y2 z2
Ç å Ç å Ç å Ç å Ç å Ç å Ç å
0 0 x1 x1 x1 0 0
3. +x= + = = + =x+ .
0 0 x2 x2 x2 0 0
Ç å
0
It is then clear that = 0R2 . Moreover, one has:
0
Ç å Ç å Ç å
x1 ≠x1 0
4. + = = 0R2 .
x2 ≠x2 0
å Ç
≠x1
One therefore sees that ≠x := . One therefore proved that (R2 , +, ·) is an abelian group.
≠x2
Besides, for all (–, —) in K2 , one can write:

ÇÇ
Ç åå Ç å å Ç å Ç å
y1 x1 –x1 + –y1 –x1 –y1
5. – · (x + y) = – · + = = + = – · x + – · y,
y2 x2 –x2 + –y2 –x2 –y2
Ç å Ç å Ç å Ç å
(– + —) · x1 –x1 + —x1 –x1 —x1
6. (– + —) · x = = = + = – · x + — · x,
(– + —) · x2 –x2 + —x2 –x2 —x2
Ç å Ç å
x1 1 · x1
7. 1 · x = 1 · = = x,
x2 1 · x2

åå Ç Ç Ç å Ç å Ç å Ç å
x1 — · x1 – · —x1 –—x1 (–—) · x1
8. – · (— · x) = – · — · =–· = = =
x2 — · x2 – · —x2 –—x2 (–—) · x2
Ç å
x1
= (–—) · = (–—) · x.
x2

We hence proved that (R2 , +, ·) is a vector space.

a
i.e. an element of K.

42 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Starting from the axioms introduced in 3.1.1, we easily deduce the following properties:

Proposition 3.1.1. 1. ’x œ E, 0 · x = 0E ,

2. ’⁄ œ K, ⁄ · 0E = 0E ,

3. ’x œ E, (≠1) · x = ≠x,

4. The following implication holds: – · x = 0E =∆ – = 0K or x = 0E .

Proof 3.1.3
To be written. ⇤

Remark 3.1.4
1. When there is no ambiguity on K one says Vector Space instead of K-Vector Space.

2. We will often write 0 instead of 0E . The same symbol will hence denotes, from one hand the
neutral element for addition in E (and in this case 0 belongs to E) and, in an other hand, the
neutral element for addition in K (and in this case 0 belongs to K).

Example 3.1.5 (Classical Vector Spaces)

1. (R, +, ·) and (C, +, ·) are both an R-Vector Space. More generally, The body (K, +, ·)
itself, where + denotes the addition between two elements of K and · denotes the ordinary
multiplication between two elements of K is a K-Vector Space. Indeed, · denotes the ordinary
multiplication between two elements of K, i.e. :
K◊K æ K
(–, x) ‘æ – · x := –x
is a K-Vector Space since (K, +, ·) ful�lls all the requirements of a Vector Space on K.
2. (Kn , +, ·) is an K-Vector Space for any n in Nú
Let n be a positive integer. De�ne + and · on Kn by setting:
(x1 , x2 , · · · , xn ) + (xÕ1 , xÕ2 , · · · , xÕn ) := (x1 + xÕ1 , x2 + xÕ2 +, · · · , xn + xÕn )
– · (x1 , x2 , · · · , xn ) := (–x1 , –x2 , · · · , –xn )

3. Let X be a set and let F(X, K) be the set of K-valued functions, de�ned on X. Then
(F(X, K), +, ·) is a K-Vector Space if the two laws + and · are de�ned by setting:

f +g : X æ K –·f : X æ K
t ‘æ f (t) + g(t) t ‘æ –f (t)

4. ({u := (un )nœN , un œ K}, +, ·) is a K-Vector Space The set of all sequences of real or
complex numbers, endowed with the addition between sequences and with the multiplication
of a sequence by a scalar is Vector Space on K.

43 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

5. (K[X], +, ·) is a K-Vector Space.

The set of polynomials with coe�cients in K is Vector Space on K.

6. (Mp,q (K), +, ·) is a K-Vector Space. The set of matrices with p rows and q columns, endowed
with the addition between two matrices and with the multiplication by a scalar is Vector Space
on K.

Remark 3.1.6
When there is no risk of confusion we say that E is a vector space instead of (E, +, ·).

3.2 Vector Subpaces

In all this section, (E, +, ·) denotes a Vector Space on K.

De�nition & Examples

It is easy to determine whether a subset of the vector space E has itself the structure of vector space,
as shows the next de�nition.

De�nition 3.2.1 (Vector Subpace)

Let F be a subset of E. (F, +, ·) is said to be a vector subspace of (E, +, ·) if:

1. 0E œ F ,

2. ’(x, y) œ F 2 , x + y œ F ,

3. ’⁄ œ K, ’x œ F, ⁄ · x œ F ,

Remark 3.2.2
It is clear that a Vector subspace is a vector space (for the laws + and · de�ned on E).

Example 3.2.3
1. R and iR are vector subspaces of the R-vector space C.

2. E and 0 are vector subspaces of E that are call trivial subspaces of E.

3. When F is a sub vector space of E then (F(F, K), +, ·) is vector subspace of (F(E, K), +, ·).

4. General theorems on continuity show that the set of continuous R-valued functions de�ned
on [0, 1] (denoted C 0 ([0, 1], R)) is a vector subspace of (F([0, 1], R), +, ·)
qn
5. The set F := {x := (x1 , x2 , · · · , xn ) œ Rn , i=1 xi = 0} is a vector subspace of Rn .
qn
6. The set F := {x := (x1 , x2 , · · · , xn ) œ Rn , i=1 xi = 1} is not a vector subspace of Rn
since it does not contain 0Rn .

7. Kn [X] := {P œ K[X], deg(P ) 6 n} is a vector subspace of K[X].

44 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

3.3 Intersection and Sum of Vector Subspaces

Intersection of Vector subspaces

Proposition 3.3.1
An intersection of Vector subspaces of E is a vector subspace of E.

Proof 3.3.2
To be written. ⇤

Proposition 3.3.3
The union of two vectors subspaces of E that are not included in each other is not a vector subspace
of E.

Proof 3.3.4
To be �lled To be written. [CPY96, p.48] ⇤

Remark 3.3.5
One can summarize the previous proposition by writing that an union of Vector subspaces is not, in
general, a vector subspace. For example the union of two lines in R2 is not a vector subspace of R2 .

De�nition 3.3.6 (Linear Combination)

Let E be a K-Vector Space and let x1 , x2 , · · · , xn be vectors of E. We call linear combination of
the vectors x1 , x2 , · · · , xn , any vector of the form:
n
ÿ
⁄i xi , where n œ Nú , and (⁄1 , ⁄2 , · · · , ⁄n ) œ Kn .
i=1

If A is a subset of E, we call linear combination of elements of A, any linear combination of a �nite

number of elements of A.

Remark 3.3.7
One might want to use sometimes the notation
ÿ
⁄a a.
aœA

This is not ambiguous if A is a �nite subset of E but this suggests, if A is an in�nite subset of E, that
only �nitely many scalars ⁄a are di�erent from 0. Hence the given sum is correctly de�ned.

De�nition 3.3.8 (Collinear vectors)

Two vectors x and y of E are said to be collinear if there exist scalars ⁄ and µ, non both equal to 0,
such that
⁄x + µy = 0.
If x ”= 0, one can not have µ ”= 0 and the condition becomes: there exists a scalar ‹ such that y = ‹x.

45 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

If A := (ai )iœI is a family of vectors of E, a linear combination of vectors of A is a sum of the form
q
iœI ⁄i ai in which all the ⁄i equal 0, except �nitely many of them. When we index a subset A by its
own elements (i.e. I = A), we recover the notation introduced at remark ??.

De�nition 3.3.9 (Span of A)

We note Span(A) the set of all linear combinations of elements of A.

Example 3.3.10
When A = {a} has a unique element, Span(A) is the set Ka = {⁄a, ⁄ œ K} of “multiples” of the
vector a. A set E := Ka, where a ”= 0 is called a line of E.

The next proposition shows, in particular, that the lines of E are vector subspaces. This can be checked
directly.

Proposition 3.3.11
Let A be a subset of E (or a family of vectors of E). The set Span(A) is a vector subspace of E.
Moreover, it is the smallest vector subspace which contains A. It is also the intersection of all vector
subspaces of E which contain A. It is called the vector subspace spanned by A and we denote it
SpanK (A) or SpanK A (or only Span(A) or Span A when there is no risk of confusion).

Proof 3.3.12
To be written [CPY96, p.49]. ⇤

Finally, one easily deduces, from Propositions 3.3.1 and 3.3.11 that SpanK (A) is the intersection of all
vector subspaces of E which contain A.

Example 3.3.13
1. If A = ÿ, then Span(A) = {0E }.

2. In the R vector space C,

• SpanR ({1}) = R,
• SpanR ({i}) = iR,
• SpanR ({1, i}) = C.

3. Every two non collinear vectors of the plan span R2 .

4. Every three non coplanar vectors of the space span R3 .

5. (1, X, X 2 , · · · , X n ) is a spanning family of Kn [X] since any polynomial with degree smaller
than n can be written under the form:
n
ÿ
ak X k with, forall k œ J0, nK, ak œ K.
k=0

We will see in Proposition 3.4.1 an easier way to prove Statement 5. of the previous example.

46 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Sum of Vector subspaces

We saw in Remark 3.3.5 that an union of Vector subspaces is not, in general, a vector subspace. The
smallest vector subspace of a vector space E which contains F and G is therefore the vector subspace
spanned by F fi G which is, in general di�erent from F fi G.

Proposition 3.3.14 (Sum of vector subspaces)

Let F and G be two vector subspaces of a vector space E. The smallest vector subspace which
contains F and G is:
H := {x + y, (x, y) œ F ◊ G}.
We call it the sum of F and G and denote it F + G.

Proof 3.3.15
To be �lled ⇤

Example 3.3.16 (Sum of vector subspaces)

1. If E = K2 then E = Span{(1, 0)}+Span{(1, 1)} since any element (x, y) of E can be written
under the form (x ≠ y)(1, 0) + y(1, 1).

2. Let F, G and H be three sub-vector spaces of a vector space E. The smallest vector subspace
which contains F, G and H is:

F + G + H := {x + y + z, (x, y, z) œ F ◊ G ◊ H}.

We call it the sum of the vector subspaces F, G and H.

3. More generally, if E1 , E2 , · · · , En are vector subspaces of E, one de�nes:

E1 + E2 + · · · + En := {x1 + x2 + · · · + xn , where ’i œ J1, nK, xi œ Ei } .

It is the smallest vector subspace of E which contains every Ei .

Hence we have, for example,

Span{(a1 , a2 , · · · , an )} = Ka1 + Ka2 + · · · + Kan .

De�nition 3.3.17 (Direct Sum of Vector Subspaces)

The sum of two vectors subspaces F and G is said to be direct if F fl G = {0E }. In this case the sum
m
F + G is denoted F G.

Proposition 3.3.18
Let F and G be two vector subspaces of a vector space E that are in direct sum. For all vector x of
m
F G there exists a unique vector y in F and a unique vector z in G such that x = y + z.

Proof 3.3.19
To be written! [CPY96, p.50]. ⇤

47 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Supplementary Vector Subspaces

De�nition 3.3.20 (Supplementary vector subspaces & Projection on a vector subspace)

Let F and G be two vector subspaces of a vector space E. We say that F and G are supplementary
m
vector subspaces of E if E = F G. In other words, F and G are supplementary vector subspaces
of E if E = F + G and if F fl G = {0E }.
In this latter case, every vector x of E can be written, in a unique way, under the form x = y + z
with y in F and z in G.
Vector y is called the projection of x, on F , parallel to G and the map p : E ‘æ F de�ned by
p(y + z) = y, for all (y, z) in F ◊ G, is called the projection of E on F , parallel to G.

Remark 3.3.21
When F and G are vector subspaces of a vector space E, to prove that F fl G = {0}, it is su�cient
to prove that x œ F fl G =∆ x = 0.

Example 3.3.22 (Supplementary vector subspaces)

1. In E := K2

• The two vector subspaces F := Span{(1, 0)} and G := Span{(0, 1)} are supplementary
vector subspaces of K2 . Indeed, any element (x, y) in K2 can be written uniquely under
the form –(1, 0) + —(0, 1), with – = x and — = y.
• The two vector subspaces F := Span{(1, 0)} and H := Span{(1, 1)} are supplementary
vector subspaces of K2 . Indeed,
• ’(x, y) œ K2 , (x, y) = (x ≠ y) · (1, 0) + y · (1, 1) = (x ≠ y, 0) + (y, y),
• If (x, 0) = (y, y), then y = 0 and thus x = 0.

2. In K3 , the vector subspaces K2 ◊{0} and Span{(0, 0, 1)} are supplementary vector subspaces.

3. In E := F(R, R), one can show that the set E of even functions and the set O are supple-
m
mentary vector subspaces (thus E O = F(R, R)).

4. In E := K[X], let P be in E\{0}. The sets

F := {Q œ E, deg(Q) < deg(P )} G := {P · Q, Q œ E}

are two supplementary vector subspaces. Use Euclidean division to prove it!

One can generalize De�nition ?? to the case of any �nite number of vector subspaces of E.

De�nition 3.3.23 (Sum of �nitely many vector subspaces)

Let E1 , E2 , · · · En be n vector subspaces of E. De�ne

E1 + E2 + · · · + En = Span(E1 fi E2 fi · · · fi En ).

By induction, one can prove that:

48 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Proposition 3.3.24
E1 + E2 + · · · + En is the set of all vectors of E which can be written under the form of a sum
x1 + x2 + · · · xn , where xi belongs to Ei , for all i in J1, nK.

Proof 3.3.25
To be written! [CPY96, p.50]. ⇤

Example 3.3.26
A family of vectors (x1 , x2 , · · · , xn ) spans E if and only if E = Kx1 + Kx2 + · · · + Kxn .

De�nition 3.3.27 (Direct Sum of Vector Spaces)

Let E1 , E2 , · · · En be n vector subspaces of E. We will say that the sum of these vector subspaces is
direct if the expansion described in Proposition 3.3.24 is unique for all vector x of E. In this latter
m m m
case we denote E1 E2 · · · En the sum of the Ei .

De�nition 3.3.28 (Plan & Hyperplan)

Let E be a vector space. We call plan of E every vector subspace which is the direct sum of two
lines of E. We call hyperplan of E every vector subspace of E which is the supplementary space
of a line in E.

3.4 Spanning Family & Linearly Independent Family of Vectors

We saw at Example 3.3.10 an explicit characterization of Span(A) when A = {a}. One can generalize
this characterization when A is �nite family of vectors of E.

Proposition 3.4.1 (Finite Spanning Family)

Let n be in Nú and A := {a1 , a2 , · · · , an } a �nite subset of E which contains n elements. The
vector subspace spanned by A is the set of all linear combinations of vectors a1 , a2 , · · · , an ; i.e.
® n
´
ÿ
Span(A) = y œ E, ÷(⁄1 , ⁄2 , · · · , ⁄n ) œ Kn such that y = ⁄i ai .
i=1

In particular, the family (a1 , a2 , · · ·, an ) spans E if

E = Span {a1 , a2 , · · ·, an }

i.e. if every element of E can be written as a linear combination of elements of the family
(a1 , a2 , · · ·, an ). In other words:
n
ÿ
’x œ E, ÷(⁄1 , ⁄2 , · · · , ⁄n ) œ Kn such that x = ⁄i ai .
i=1

Proof 3.4.2
Obvious in view of De�nitions 3.3.6 and 3.3.9. ⇤

49 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Remark 3.4.3
1. If a family of vectors spans E, any family of vectors obtained by permuting its elements also
spans E. It does not depend on the order of the vectors which belong to the spanning family.

2. Note that proving Statement 5. of Example 3.3.13 is now obvious.

3. When the family of vectors one considers in in�nite one can not use Proposition 3.4.1 anymore.
One has to come back to De�nition 3.3.9.

Example 3.4.4
A family of vectors (x1 , x2 , · · · , xn ) spans E if and only if E = Kx1 + Kx2 + · · · + Kxn .

Proposition 3.4.5
Let G be a family of vectors which spans a vector space E. A family X of elements of E spans E if,
and only if, every element of G is a linear combination of elements of X.

Proposition 3.4.6
Let F and G be two vector subspaces of E. A vector x belongs to Span(F fi G) if and only if there
exists a vector y of F and a vector z of G such that x = y + z.

Proof 3.4.7
To be written [CPY96, p.50]. ⇤

The next de�nition derives naturally from De�nition 3.3.6.

De�nition 3.4.8 (Linear Independence)

Let (xi )iœJ1,nK be a �nite family of elements of E.

1. We say that (xi )iœJ1,nK is a linearly independent family of vectors of E if, for all family
(⁄i )iœJ1,nK in Kn , the following implication holds:
n
ÿ
⁄i xi = 0 =∆ ⁄1 = ⁄2 = · · · ⁄n = 0.
i=1

2. In case the previous implication does not hold, i.e. if one can �nd a family (⁄i )iœJ1,nK in Kn such
q
that not all the ⁄i equal 0, and such that ni=1 ⁄i xi = 0, one says that the family (xi )iœJ1,nK is
linearly dependent or not linearly independent.

Remark 3.4.9
If a family of vectors is linearly independent, any subset of vectors of this family obtained by per-
muting its elements is also linearly independent. The linear independence of a family of vectors does
not depend on the order of its elements.

50 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Example 3.4.10 (Linear Independence)

Let

1. A family with only one element, denoted x, is linearly independent if and only if x ”= 0.
Indeed,

• If x = 0, then 1 · x = 0 and 1 ”= 0, and thus the family (x) is not linearly independent.
• If x ”= 0, then ⁄ · x = 0 =∆ ⁄ = 0.

2. In the R-vector space C, the family (1, i) is linearly independent since:

’(a, b) œ R2 , a + ib = 0 =∆ a = b = 0.

3. In K3 vectors e1 := (1, 0, 0), e2 := (0, 1, 0) and e3 := (0, 0, 1) form a linearly independent

family of vectors. Indeed,

’(⁄1 , ⁄2 , ⁄3 ) œ K3 , ⁄1 e1 + ⁄2 e2 + ⁄3 e3 = 0 =∆ (⁄1 , ⁄2 , ⁄3 ) = (0, 0, 0)

=∆ ⁄1 = ⁄2 = ⁄3 = 0.

4. More generally, in Kn , the vectors e1 := (1, 0, · · · , 0), e2 := (0, 1, 0, · · · , 0), · · · , en :=

(0, · · · , 0, 1) form a linearly independent family of vectors since:

’(⁄1 , · · · , ⁄n ) œ Kn , ⁄1 e1 + · · · + ⁄n en = 0 =∆ (⁄1 , ⁄2 , · · · , ⁄n ) = (0, 0, · · · , 0)

=∆ ⁄1 = ⁄2 = · · · = ⁄n = 0.

5. The family (1, X, X 2 , · · · , X n ) is linearly independent in K[X] since the polynomial

qn
k=0 ⁄k X equals 0 if and only if all ⁄k equal 0.
k

6. The empty family, i.e. with no vector in it, is linearly independent since there is no linear
combination with all non zero coe�cient which equals 0.

Proposition 3.4.11
1. Every subset of vectors of a linearly independent family of vectors is linearly independent.

2. Every set of vectors containing a non linearly independent family of vectors is not linearly
independent.

Proof 3.4.12
To be written [CPY96, p.50]. ⇤

Remark 3.4.13
1. A linearly independent family of vectors can not contain the null vector.

2. A linearly independent family can not have two proportional vectors.

51 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Proposition 3.4.14
Let (xi )iœJ1,nK be a linearly independent family of vectors of E. The family of vectors
(x1 , x2 , · · · , xn , x) is linearly dependent if and only if x is a linear combination of x1 , x2 , · · · , xn .

Proof 3.4.15
To be written. ⇤

Theorem 3.4.16
Let (xi )iœJ1,nK be a family of linearly independent vectors of E and let (⁄1 , · · · , ⁄n ) and (µ1 , · · · , µn )
be two family of scalars, we have the following implication:
n
ÿ n
ÿ
⁄ i xi = µi xi =∆ ’i œ {1, 2, · · · , n}, ⁄i = µi .
i=1 i=1

Proof 3.4.17
To be written. ⇤

3.5 Linear Maps

Before beginning this section, readers should be familiar with the notions of injective, sur-
jective and bijective of maps (also called one-to-one, onto and one-to-one correspondence
maps). See Appendix A if needed.

Let E, F and G be three vector spaces on K.

De�nition and characterization

De�nition 3.5.1 (Linear Map)

An application u : E æ F is said to be a linear map (also called a linear mapping, linear transfor-
mation or linear function) if it preserves the two laws of a vector space:

’(x, y) œ E 2 , u(x + y) = u(x) + u(y) & u(⁄x) = ⁄u(x) (5.1)

These two equalities can be gathered under the form:

’(x, y) œ E 2 , u(⁄x + y) = ⁄u(x) + u(y) (5.2)

We also say that u is a morphism between vector spaces E and F . We say that u is an:

• endomorphism if E = F ,

• isomorphism if u is bijective,

• automorphism if u is bijective and if E = F .

Notations The set of linear maps from E to F is denoted L(E, F ) or LK (E, F ) if one wants to
precise the body K. Moreover, L(E) := L(E, E) denotes the set of endomorphisms of E.

52 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Remark 3.5.2
If u : E æ F is a linear map then

1. we have u(0E ) = 0F since:

u(0E ) = u(0R · 0E ) = 0F .

2.
’x œ E, u(≠x) = ≠u(x).

3. We immediately can prove (by induction on n in N) that, for all (x1 , x2 , · · · , xn ) in E n and
(⁄1 , ⁄2 , · · · , ⁄n ) in Rn :
n
ÿ n
ÿ
u( ⁄ i xi ) = ⁄i u(xi ).
i=1 i=1

Example 3.5.3 (Linear Map)

1. The linear maps from R to R are the maps x ‘æ kx, with k in R. (Try to prove it).

2. For any (a, b) in R2 , the map u : R2 æ R is linear.

(x, y) ‘æ ax + by
3. For any (a, b, c, d) in R4 , the map u : R2 æ R2 is an endomorphism of R2 .
(x, y) ‘æ (ax + by, cx + dy)

4. The di�erentiation is a linear map from C 1 (R) in C 0 (R). This means that:

 : C 1 (R) æ C 0 (R)
f ‘æ f Õ

Note in particular that the restriction of  to C Œ (R) is an endomorphism of C Œ (R).

5. For all (a, b, c) in R3 , the map:

: C 2 (R) æ C 0 (R)
f ‘æ af ÕÕ + bf Õ + cf

is a linear map.

6. If F and G are supplementary vector subspaces of E, one can check that the map, from E to
F , “projection on F , parallel to G” is linear.

De�nition 3.5.4 (Linear form)

We call linear form any linear map from E to R.

Example 3.5.5 (Linear form)

1. For any (–, —) in R2 , the map f : R2 æ R is a linear form on R2 .
(x, y) ‘æ ax + by

2. On the segment [a, b], the Riemann Integral is a linear form on C 0 (I). Indeed, it is easy to

53 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

verify that:
: C 0 (I) æ R is linear.
sb
f ‘æ a f (t) dt
is linear.

3. For any – in R, the map P ‘æ P (–) is a linear form on R[X].

Structure of L(E, F ) and L(E).

Proposition 3.5.6
Let f and g be two endomorphisms of E and – and — be two scalars. Then the map –f + —g is linear.

Proposition 3.5.7
1. If f œ L(E, F ) and g œ L(F, G) then g ¶ f belongs to L(E, G).

2. If f is an isomorphism from the vector spaces E and F , then f ≠1 is linear.

3. (L(E, F ), +, ·) is a vector space.

Proof 3.5.8
To be written! [CPY96, Proposition II.17 p.52]. ⇤

Proposition 3.5.9 (Properties of composition)

Let u œ L(E, F ) and v œ L(F, G) and (–, —) in R2 .

1. Let Ï and  be two linear applications from F to G, we have:

(–Ï + —Â) ¶ u = –(Ï ¶ u) + —(Â ¶ u).

2. Let Ï and  be two linear applications from E to F , we have:

v ¶ (–Ï + —Â) = –(v ¶ Ï) + —(v ¶ Â).

In other words, the maps:

L(F, G) æ L(E, G) L(E, F ) æ L(E, G)

and
Ï ‘æ Ï ¶ u Ï ‘æ v ¶ Ï

are linear.

3. (L(E), +, ¶) is non commutativea in general. In other words, for u and v in L(E), u¶v ”= v¶u
in general .
a
For those who have notions of algebra, one says that (L(E), +, ¶) is a non commutative ring.

54 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

De�nition 3.5.10 (Linear group)

We call GL(E) the set of automorphisms of the vector space E.

Proposition 3.5.11
(GL(E), ¶) is a group. We call it the Linear Group of E.

Kernel and Image of a linear map

Recall
For any sets E and F and any map u : E æ F , we call:

• The image by u of a subset E Õ of E the set:

u(E Õ ) := {u(x), x œ E Õ },

• The preimage by u of a subset F Õ of F the set:

u≠1 (F Õ ) := {x œ E, u(x) œ F Õ }.

Proposition 3.5.12
Let u : E æ F be a linear map.

• If E Õ is vector subspace of E, then u(E Õ ) is a vector subspace of F .

• If F Õ is vector subspace of F , then u≠1 (F Õ ) is a vector subspace of E.

De�nition 3.5.13 (Kernel & Image of a linear map)

Let u : E æ F be a linear map. We call:

1. Kernel of u (or null space of u) the set de�ned by:

Ker(u) := u≠1 ({0F }) = {x œ E, u(x) = 0F },

2. Image of u the set de�ned by:

Im(u) := u(E) = {u(x), x œ E},

Remark 3.5.14
Note that one can also denote N (u) the kernel of u.

Proposition 3.5.15
Let u : E æ F be a linear map. Then the set Ker(u) is a vector subspaces of E and the set Im(u) is
a vector subspaces of F .

55 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Proof 3.5.16
To be �lled ⇤

between .tex

Theorem 3.5.17

Proof 3.5.18
To be �lled ⇤

Remark 3.5.19
1. When one knows that a map is linear, in order to show that it is injective one usually uses
Property 2. or its translation 3.

2. Let u : E æ E Õ be a linear map.

• If F and G are two vector subspaces of E, one has:

u(F + G) = u(F ) + u(G).

• More generally, if E1 , E2 , · · · , En are vector subspaces of E then

u(E1 + E2 + · + En ) = u(E1 ) + u(E2 ) + · · · + u(En ).

3.6 Basis

De�nition 3.6.1 (Basis of a Vector Space)

A family of elements of E is a basis of E if it is a linearly independent family which spans E.

The following example gives basis of some classical vector spaces. The linear independence of the
family of vectors presented below has been established in Example ??, on page ??.

Example 3.6.2 (Basis of a Vector Space)

1. (1, i) constitutes a basis of the R-vector space C.

2. The family constituted with vectors e1 := (1, 0, 0), e2 := (0, 1, 0) and e3 := (0, 0, 1) forms a
basis of K3 . Note that B := (e1 , e2 , e3 ) is called the standard basis of K3 . Moreover, any
vector x := (x1 , x2 , x3 ) of K3 can be written as x = x1 e1 + x2 e2 + x3 e3 . Thus (e1 , e2 , e3 )

3. More generally, we call standard basis of Kn the family of vectors (e1 , e2 , · · · , en ) where,
for every k in J1, nK, ek := (0, 0, · · · , 0, 1, 0, · · · , 0) i.e. all entries are 0 except for the k th
which equals 1.
qn
4. The family (1, X, X 2 , · · · , X n ) is a basis of Kn [X] since every single polynomial k=0 ⁄k Xk
equals 0 if and only if all ⁄k equal 0. it is called the standard basis of Kn [X].

5. The family (X k )kœN is a basis of the vector space of polynomials.

56 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

6. Let p and q be two positive integers. Denote Ei,j the p ◊ q matrix the coe�cients of which
are all zeros except for the one on the ith row and j th column which equals 1. Let A :=
(ai,j )(i,j)œJ1,pK◊J1,qK be a matrix in Mp,q (K). We clearly have the equality:
p ÿ
ÿ q
A= aij Ei,j .
i=1 i=1

One then deduces that the family (Ei,j )(i,j)œJ1,pK◊J1,qK is a basis of Mp,q (K).

Remark 3.6.3
The two following formula might be of interest for some exercises we will perform later.
®
2 (0) if j ”= k
’(i, j) œ J1, pK , t
Ei,j = Ej,i & Ei,j · Ek,l =
Ei,l if j = k

Theorem 3.6.4 (Unicity of the expansion in a given basis)

A family of vectors (ei )iœJ1,nK is a basis of E if, and only if, for all vectors x in E, there exists a
unique family (⁄i )iœJ1,nK of Kn such that:
n
ÿ
x= ⁄i ei .
i=1

The family (⁄i )iœJ1,nK is called family of components or coordinates of x in the basis B := {ei , i œ
J1, nK}.

Proof 3.6.5
To be �lled ⇤

Remark 3.6.6
For any given family family of vectors (x1 , x2 , · · · , xn ) of E, one can easily prove that the map:

Kn æ E
qn
(⁄i )iœJ1,nK ‘æ i=1 ⁄i xi

is linear. Moreover,

• it is injective (or one to one) if, the family (xi )iœJ1,nK is linearly independent.

• it is surjective (or onto) if, the family (xi )iœJ1,nK spans E.

• it is bijective if, the family (xi )iœJ1,nK is a basis of E.

57 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

Basis and Linear Maps

Proposition 3.6.7
Let u : E æ F be a linear map and let A := (ai )iœI be a family of vectors of E.

1. If u is injective then the family A is linearly independent in E if and only if (u(ai ))iœI is
linearly independent in F .

2. If u is surjective then the family A spans E if and only if (u(ai ))iœI spans F .

3. If u is bijective then the family A is a basis of E if and only if (u(ai ))iœI is a basis of F .

Proposition 3.6.8
To be written!

Proposition 3.6.9 (Link between linear map and basis)

Let E and F be two vector spaces. Let A := (ai )iœI be a basis of vectors of E and B := (bi )iœJ be
a family of vectors of F . There exists a unique linear map u from E to F such that:

’i œ I, u(ai ) = bi .

Moreover,

u is injective if, and only if, B is linearly independent.

u is surjective if, and only if, B spans F .

u is bijective if, and only if, B is a basis of F .

3.7 Linear Equations

De�nition & Examples
In Theorem 2.6.14, we saw how to solve a square linear system, using matrices. We here, present a
way to think about linear system in general (i.e. not only the square ones) in terms of linear maps.

De�nition 3.7.1 (Linear Equation)

A linear equation is an equation of the form u(x) = b, where:

• u : E æ F is a linear map and E and F are vector spaces,

• v is vector of F ,

• the unknown, x, is a vector of E.

Example 3.7.2 (Linear Equation)

1. The linear system given, at the beginning of Section 2.1, under the form of a systems of equa-
tions can - of course! - be written under the form u(x) = b. In this latter case, E = Rq and
F = Rp .

58 S����� 2023
L����� A������ Lecture Notes C������ 3: Vector Spaces

2. A linear di�erential equation of the �rst order, like:

a(t)y Õ (t) + b(t)y(t) = c(t) (7.3)

where the functions a, b and c are all given and continuous over an interval I, and where the
unknown is the map y, is a linear system. Indeed, one just have to de�ne u in the following
manner:
u : C 1 (I) æ C 0 (I)
y ‘æ a.y Õ + b.y
Equation (7.3) can then be written as:

u(y) = c.

Structure of the Set of solutions

Let u be a linear map, from a vector space E to a vector space F , and let b be an element of F . Let’s
consider the following linear equation:
u(x) = b. (E)
Denote S(E) the set of all solutions of (E) and S(E0 ) the set of all solutions of:

u(x) = 0F . (E0 )

Equation (E0 ) is called homogeneous equation (or equation without second member) associated to
(E). We have the following result.

Proposition 3.7.3 (Structure of the solutions of a linear equation)

1. S(E0 ) is a vector subspace of E. In particular it is nonempty.

2. If S(E) ”= ÿ, and if x0 is an element of S(E) , then the set of all solutions of (E) is:

S(E) = {x0 + h, h œ S(E0 ) }.

In other words, the set of all solutions of (E0 ), if it is not empty, is an a�nea subspace of E.
a
This geometric notion will be speci�ed later in this course.

Proof 3.7.4
To be �lled: To be �lled! p.810 ⇤

Example 3.7.5
To be �lled!

59 S����� 2023
C������ 4

Vector Spaces with �nite Dimension

In all this chapter, n is a positive integer and E denotes a vector space on C. Moreover K denotes the
set R of real numbers.

De�nition 4.0.1 (Finite Dimension)

A Vector space E is said to be �nite-dimensional, if it admits a �nite spanning family. Otherwise, we
say that E is in�nite-dimensional.

4.1 First Results

Proposition 4.1.1
In a vector space E which has a spaning family with n elements, every family that has n+1 elements
is not linearly independent.

Proof 4.1.2
To be written [CPY96, Proposition II.37 p.57]. ⇤

Theorem 4.1.3 (De�nition of the dimension of a Vector Space)

A vector space with �nite dimension has basis. They all have the same (�nite) number of elements.
This number is called the dimension of the vector space E and is denoted dim E (or dimK E if one
wants to specify the body K).

Proof 4.1.4
To be written [CPY96, Proposition II.38 p.58]. ⇤

Proposition 4.1.5
Let E be a vector space with �nite dimension n.

1. Every linearly independent family of vectors of E has at most n elements.

2. Every spanning family of vectors of E has at least n elements.

3. Every linearly independent family of vectors of E, which has n elements is a basis of E.

60
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

Proof 4.1.6
To be written [CPY96, Proposition II.39 p.58]. ⇤

Remark 4.1.7
According to the previous proposition, if (x1 , x2 , · · · , xp ) is a linearly independent family and if
(y 1 , y 2 , · · · , y q ) is spanning family of E, then p 6 q.
Thus every spanning family has, at least, as many elements as any linearly independent family. We
therefore deduce the two following results.

Theorem 4.1.8 (De�nition of the dimension of a vector space)

Let E be a �nite-dimensional vector space, all basis of E have the same number of elements, denoted
n. The positive integer n is called dimension of E on K, or more simply dimension of E.

Proof 4.1.9
to be �lled ⇤

Example 4.1.10 (De�nition of the dimension of a vector space)

1. The vector space {0} has a dimension equal to 0.

2. K has a dimension equal to 1 on K.

3. C has a dimension equal to 2 on R, since (1, i) is a basis of C.

4. Kn is a vector space with dimension equals to n, since a basis of it is constituted by:

e1 := (1, 0, · · · , 0), e2 := (0, 1, · · · , 0), ··· , en := (0, · · · , 0, 1),

5. Kn [X] is a vector space with dimension equals to n + 1 on K, one basis of Kn [X] is

(1, X, X 2 , · · · , X n ).

Corollary 4.1.11
Let E be a �nite dimensional vector space of dimension n and let (x1 , x2 , · · · , xp ) a family with p
elements of E.

• If p > n, the family (x1 , x2 , · · · , xp ) is not linearly independent.

• If p < n, the family (x1 , x2 , · · · , xp ) does not span E.

Example 4.1.12
To be �lled!!!

61 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

4.2 Dimension of Vector Subspaces

Theorem 4.2.1 (Dimension of Vector Subspaces)

Let E be a �nite-dimensional vector space. Let F be a vector subspace of E. Then the vector space
F is �nite dimensional. Moreover we have the the relation dim F 6 dim E. Moreover F equals E
if, and only if, dim F = dim E.

Proof 4.2.2
To be written [CPY96, Theorem II.40 p.59]. ⇤

Proposition 4.2.3 (In�nite dimensional vector space)

A vector space E is in�nite-dimensional if, and only if, there exists a sequence (xn )nœN of elements
of E such that for all n in N, the family of vectors (xk )06k6n is linearly independent.

Proof 4.2.4
To be �lled ⇤

Example 4.2.5
Since the family of vectors (1, X, X 2 , · · · , X n ) is linearly independent, for all n in N, Theorem 4.1.8
allows one to state that the vector space K[X] is in�nite-dimensional.

Complete a Family of Vectors into a Basis

Theorem 4.2.6 (Completion of a family of vectors)

Let E be a �nite-dimensional vector space with n := dim E. Let A be a family of vectors which
spans E and let B := (b1 , · · · , bp ) be a linearly independent family of vectors of E. There exists
vectors, denoted ap+1 , · · · , an of A such that the family of vectors (b1 , · · · , bp , ap+1 · · · , an ) is a
basis of E.

Proof 4.2.7
To be written [CPY96, Theorem II.42 p.59]. ⇤

Corollary 4.2.8
Let E be a �nite-dimensional vector space with n := dim E.

1. From every spanning family A of E one can extract a basis of E.

2. Every linearly independent family of E can be completed so that it becomes a basis of E (in
general there are in�nitely many ways to do so).

3. Every spanning family of E with n elements is a basis of E.

Proof 4.2.9
To be �lled! See [CPY96, p.59 Corolaire II.43] ⇤

62 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

Remark 4.2.10
If E = {0}, the only linearly independent family is the empty family which, in this case, the only
basis of E.

Theorem 4.2.11 (Family of vectors and basis)

Let B be a family of elements of a vector space E of �nite dimension n. The following statements
are equivalent:

(i) B is a basis of E,

(ii) B is a linearly independent family of E with n elements.

(iii) B is a family of n vectors which spans E.

Proof 4.2.12
To be �lled! ⇤

Remark 4.2.13
This fundamental result is of a great use to show that a family B is a basis of a vector space the
dimension of which is known. In most cases we prove that B is a linearly independent family with
n elements.

Example 4.2.14
1. In the R vector space C the family (1, Ê) is linearly independent, i.e. is a basis, if and only if,
Ê is not a real number.

2. Two non-colinear vectors of a vector space of dimension 2 form a basis.

Example 4.2.15 (Linear recurrence relation of order 2)

Linear recurrence relation of order 2

Example 4.2.16 (Spanning and linearly independent families of vectors)

1. In E = R4 , denote the vector space
F := {(x1 , x2 , x3 , x4 ), x1 + x2 + x3 + x4 = 0}.
A vector x := (x1 , x2 , x3 , x4 ) of F is characterized by its �rst three elements x1 , x2 and x3 .
This leads us to think that dim(F ) = 3. One can e�ectively prove that dim(F ) = 3 by giving
the three following vectors of F :
f 1 := (1, 0, 0, ≠1), f 2 := (0, 1, 0, ≠1), f 3 := (0, 0, 1, ≠1).
Let us prove that F := (f 1 , f 2 , f 3 ) is a basis of F by showing that:
a) F is a linearly independent family of vectors. Let (⁄1 , ⁄2 , ⁄3 ) be in R3 .
⁄1 f 1 + ⁄2 f 2 + ⁄3 f 3 = 0 =∆ (⁄1 , ⁄2 , ⁄3 , ≠⁄1 ≠ ⁄2 ≠ ⁄3 ) = 0
=∆ ⁄1 = ⁄2 = ⁄3 = 0.
This last implication establishes that F is a linearly independent family of vectors.

63 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

b) F spans F . Let x := (x1 , x2 , x3 , x4 ) be an element of F . By de�nition of F , on has the

equality x = (x1 , x2 , x3 , ≠x1 ≠ x2 ≠ x3 ). Thus one can write x = x1 f 1 + x2 f 2 + x3 f 3 .
This latter equality, as well as the arbitrary choice of x in F proves that F spans F .

2. In E = R4 , denote the vector space

G := {(x1 , x2 , x3 , x4 ), x1 + x2 = 0 and x3 + x4 = 0}.

Any vector x := (x1 , x2 , x3 , x4 ) of G ful�lls the equality x = (x1 , ≠x1 , ≠x4 , x4 ). This leads
us to think that dim(G) = 2. Let’s prove it by showing that the family of vectors G := (g 1 , g 2 )
de�ned by by setting:

g 1 := (1, ≠1, 0, 0), g 2 := (0, 0, ≠1, 1)

constitutes a basis of G.

a) G is a linearly independent family of vectors. Let (⁄1 , ⁄2 ) be in R2 .

⁄1 g 1 + ⁄2 g 2 = 0 =∆ (⁄1 , ≠⁄1 , ≠⁄4 , ⁄4 ) = 0

=∆ ⁄1 = ⁄4 = 0.
This last implication establishes that G is a linearly independent family of vectors.
b) G spans G. Let x := (x1 , x2 , x3 , x4 ) be an element of G. By de�nition of G, on has the
equality x := (x1 , ≠x1 , ≠x4 , x4 ). Thus one can write x = x1 g 1 + x4 g 2 . This latter
equality, as well as the arbitrary choice of x in G proves that G spans G.

3. Since K is a 1-dimensional vector space on K, its vector subspaces can only have a dimension
equal to 0 or 1. Thus K has onmy to vector subspaces that are {0} and K.

Proposition 4.2.17
Any vector subspace F of a �nite dimensional vector space E has, at least one supplementary vector
m
space G. Thus F G = E.

Proof 4.2.18
To be �lled! See [CPY96, p.59 Proposition II.44] ⇤

Remark 4.2.19
Unless F equals {0E } or E, a vector subspace has many supplementary vector subspaces. It is
therefore very important to speak about a supplementary vector subspace.

64 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

4.3 Link Between Dimensions

Dimension and Isomorphism

Proposition 4.3.1
Let E be a �nite-dimensional K-vector space of dimension n. A vector space F is isomorphic to E
if and only if F is a �nite-dimensional vector space of dimension n.

Proof 4.3.2
To be �lled! ⇤

Corollary 4.3.3
Any K-vector space of dimension n is isomorphic to Kn .

Proof 4.3.4
To be �lled ⇤

Dimension of a Product of Vector Spaces

Proposition 4.3.5 (Dimension of a Product of Vector Spaces)

Let E and F be two �nite dimensional vector spaces, then E ◊ F is a �nite dimensional vector space,
the dimension of which is
dim(E ◊ F ) = dim(E) · dim(F ).

Proof 4.3.6
To be �lled ⇤

Example 4.3.7
Donner l’exemple de Mp,q (R) ƒ Rp ◊ Rq !

Dimension of a Sum of Vector Spaces

Proposition 4.3.8 (Dimension of a Sum of Vector Spaces)

Let E be a K-vector Space of �nite dimension. Let F and G be two vector subspaces of E, we have
the following equality:

dim(F + G) = dim(F ) + dim(G) ≠ dim(F fl G).

Proof. To be �lled! See [CPY96, p.60 Proposition II.45] ⇤

Example 4.3.9
Expand [CPY96, p.62 Example II.46]

65 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

4.4 Rank-Nullity Theorem

Rank of Family of Vectors, of a Linear Map

De�nition 4.4.1 (Rank of a family of vectors)

Let F be a �nite family of vectors of E. The rank of F, denoted rk(F) or rk F, is the dimension of
the vector subspace of E spanned by F.

Remark 4.4.2
Let F := (x1 , x2 , · · · , xn ) be a family of vectors of E.

1. Since F is a spanning family of Span F, one can extract a basis of Span F. Thus rk F 6 n
and the equality holds if and only if F is a linearly independent family.

2. If F has r linearly independent vectors, we have rk F 6 r, and the equality case happens
if and only if these r linearly independent vectors span Span F, in other words the equality
happens if and only if any element of F is a linear combination of these r vectors.

Example 4.4.3 (Rank of family of vectors)

Let us consider the following vectors of R3 :

f 1 := (1, 0, 0), f 2 := (0, 1, 0), f 3 := (0, 0, 1), f 4 := (1, 1, 1). f 5 := (1, 0, 1).

Denote F := (f 1 , f 2 , f 3 ), G := (f 1 , f 3 , f 5 )and H := (f 1 , f 2 , f 3 , f 4 , f 5 , ). It is easy to see that:

(i) since (f 1 , f 2 , f 3 ) is a linearly independent family of vectors of R3 , one has the equality
dim(Span(F)) = 3. Thus rk(F) = 3

(ii) since f 1 + f 3 = f 5 the family of vectors G is not linearly independent. Thus, one can write:
dim(Span(G)) < 3. Moreover, since (f 1 , f 3 ) is a linearly independent family of vectors,
one has dim(Span(G)) > 2. One therefore conclude that dim(Span(G)) = 2 and thus, that
rk(G) = 2.

(iii) since F µ H, one has Span(F) µ Span(H) µ R3 and thus 3 = dim(Span(F)) 6

dim(Span(H)) 6 3. In other words Span(H) = R3 and thus rk(H) = 3.

De�nition 4.4.4 (Rank of a linear map)

Let E and F be two K vector spaces and u : E æ F be a linear map. We call rank of u, and we
denote it rk(u), the dimension of Im(u), when it is �nite.

Remark 4.4.5
• If F is a �nite-dimensional space with dimension n, the rank of any linear map u : E æ F is
�nite and smaller or equal to n. moreover, the equality holds if and only if u is surjective.

• Let E be a �nite-dimensional vector space with dimension p and B := (e1 , e2 , · · · , ep ) a basis

of E. If u : E æ F is a linear map, we saw at Proposition ?? that Im(u) is spanned by the
family of vectors (u(ek ))16k6n . This proves that rk(u) is �nite. It is equal to rk(u(B)). Thus
is smaller or equal to p and is equal , if and only if, u(B) is a basis of Im(u) (i.e. if and only if

66 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

u is injective).

Rank-Nullity Theorem

Theorem 4.4.6 (Rank-Nullity Theorem)

Let E be a �nite-dimensional vector space and a u : E æ F a linear map valued in a vector space
F . Then Im(u)is a �nite-dimensional vector space. moreover one has the following equality:

dim(E) = dim(Ker(u)) + dim(Im(u)). (4.1)

Remark 4.4.7
In view of De�nition 5.3.12, the previous equality can be rewritten under the form

dim(E) = dim(Ker(u)) + rk(u). (4.2)

Example 4.4.8 (Rank-Nullity Theorem)

1. Let h : R2 æ R2 be the linear map de�ned, for all (x, y) in R2 , by setting:

h(x, y) := (x ≠ y, ≠3x + 3y).

It is clear that Ker(h) = Span{(1, 1)}. According to the rank nullity theorem we therefore
know that dim(Im(h)) = 2 ≠ 1 = 1. Thus Im(h) = Span{h((1, 0))} = Span{(1, ≠3)}.

2. Let be the endomorphism of Rn [X] de�ned by:

(P ) = P (X + 1) ≠ P (X).

• Every constant polynomial is in the kernel of . Conversely, if (P ) = 0, then P (X +

1) = P (X) and thus all integers are roots of the polynomial P (X) ≠ P (0). It is therefore
null and thus P is constant. This implies that Ker = R0 [X].
• According to the rank nullity theorem, dim(Im( )) = n. Since Im( ) µ Rn≠1 [X], the
dimension of which is also n, we deduce that Im = Rn≠1 [X].

From these two points one can deduce that every polynomials with a degree smaller than n is
of the form P (X + 1) ≠ P (X), where deg(P ) 6 n

Corollary 4.4.9
Two �nite-dimensional vector spaces E and F have the same dimension if and only if there exist a
bijective linear map u : E æ F .

Proof 4.4.10
To be �lled! See [CPY96, p.62 Corollaire II.48] ⇤

Remark 4.4.11
Let u : E æ F be a bijective linear map. Its inverse function, denoted u≠1 , is also linear.

67 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

Linear Maps and Finite Dimension

Corollary 4.4.12 (Characterization of Isomorphisms)

Let E and F be two �nite-dimensional vector spaces with the same dimension, denoted n and let
u : E æ F be a linear map. The following properties are equivalent;

(i) u is injective (or one to one),

(ii) u is surjective (or onto),

(iii) u is bijective (i.e. both one to one and onto).

Proof 4.4.13
To be �lled! See [CPY96, p.62 Corollaire II.50] ⇤

Example 4.4.14
To be written!

Corollary 4.4.15
If u is an endomorphism, of a �nite-dimensional vector space E, it is equivalent to to say that the
map u is injective, or surjective or bijective.

Proof 4.4.16
Obvious in view of the previous Corollary. ⇤

Remark 4.4.17
Be careful if E is not a �nite-dimensional vector space!
• An endomorphism of E may be injective without being surjective, as the example of the fol-
lowing linear map shows:
: R[X] æ R[X]
P ‘æ X · P (X).
Indeed, is clearly linear. Moreover, Ker( ) = {P œ R[X], X · P (X) = 0} = {P :=
qq q+1 s.t. qq
k=0 ak X , q œ N, (a0 , · · · , aq ) œ R
k k+1 = 0}. Thus Ker( ) = {0} and
k=0 ak X
therefore is injective. However P (X) := 1 does not belong to Im( ) which is therefore not
surjective.
• An endomorphism of E may be surjective without being injective, as the example of the fol-
lowing linear map shows:
: R[X] æ R[X]
P ‘æ P Õ (X).
Indeed, any constant polynomial has an image by which equals 0 so is not injective.
q
Besides any polynomial P := qk=0 ak X k of R[X] ful�lls the equality
Ç q å
ÿ ak
X k+1
= P.
k=0
k+1
This makes surjective.

68 S����� 2023
L����� A������ Lecture Notes C������ 4: Vector Spaces with �nite Dimension

4.5 The four fundamental subspaces

To be �lled (See Strang the 4 pages in Ressources/ books Linear Algebra)

69 S����� 2023
C������ 5

Use of Basis

In this chapter, we show how the language of Vector Spaces can be translated into Matrix language
and vice-versa. More precisely, we are considering �nite-dimensional vector spaces and we choose a
basis for each of them. We �rst show that the objects de�ned in Chapters 3 and 4 (vectors spaces and
linear maps) have a matrix representation. Then we study how these representations depend on the
choice of these basis.

5.1 Presentation of the problem

Denote e := (e1 , e2 , e3 ) the standard basis of R3 . Let f := (f 1 , f 2 , f 3 ) be the family of vectors
de�ned by setting: 8
< f 1 := e1 + e3
>
f 2 := e2
>
:
f 3 := e3 .
Is is easy to verify that the vectors of f are linearly independent. Therefore f is a basis of R3 .
Let x be the vector of R3 , de�ned by setting:

x := 2e1 + 3e2 ≠ 4e3 . (1.1)

So far, in this case, we used the following notation:

Ö è
2
x := 3 . (1.2)
≠4

Besides, one can verify that:

x = 2f 1 + 3f 2 ≠ 6f 3 .
Thus, according to our previous habit, one could write:
Ö è
2
x= 3 . (1.3)
≠6

The only problem with this kind of notation is that it is leading to a contradiction since the same
quantity, the vector x, has two di�erent values, according to (1.2) and (1.3). However, De�nition 1.1 is

70
L����� A������ Lecture Notes C������ 5: Use of Basis

valid since it does de�ne, in a unique way, vector x. In order to overcome this apparent contradiction,
we must precise, in which basis the coordinates of a given vector are speci�ed.
More generally, de�ne the endomorphism of R3 , denoted u, by setting:
8
< u(e1 ) := 2e1 ≠ 3e2 + 2e3
>
u(e2 ) := ≠3e1 ≠ 6e2 + 6e3
>
:
u(e3 ) := 2e1 ≠ 2e2 + 2e3 .

How can we express simply u(f 1 ), u(f 2 ), u(f 3 ), and, more generally u(y), for any vector y :=
af 1 + bf 2 + cf 3 , where a, b and c are real numbers?
Some other questions arise:

- How the matrix representations of the same endomorphism u, in di�erent basis of E, are linked
to one another?

- Should one use preferably one basis over another, to perform computations?

All these questions will �nd an answer in this chapter.

5.2 Matrix Representation

Vectors
Let E be a �nite-dimensional vector space with dim(E) = p and let e := (e1 , · · · , ep ) be a basis of
E. For every vector x of E, we have a unique expansion
p
ÿ
x= –i ei (2.4)
i=1

where the –i are scalar (they are the coordinates or components) of vector x, in the basis e. We write:
â ì
–1
[x]e = .. .
.

–p

In other words, [x]e is the column matrix of the coordinates of vector x, in the basis e. We say that
the column matrix represents the vector x in the basis e.
More generally, if x1 , · · · xq are vectors of E, we denote ([x1 ]e · · · [xq ]e ) or simply [x1 , · · · , xq ]e the
p ◊ q matrix, the j th column of which is [xj ]e .
We easily verify that the map x ‘æ [x]e , from E to Mp,1 (K), is linear, i.e. , for all x and y in E and ⁄
in K, we have [⁄x + y]e = ⁄ [x]e + [y]e . Moreover, since this map transforms the basis e of E into
the standard basis of Mp,1 (K), it is bijective (according to Proposition 3.6.7).

Linear Maps
Let E be a �nite-dimensional vector space with dim(E) = p and let F be a �nite-dimensional vector
space with dim(F ) = q. Let e := (e1 , · · · , ep ) be a basis of E, f := (f 1 , · · · , f q ) be a basis of F and

71 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

u : E æ F be a linear map. We denote ⁄ij (1 6 i 6 q) the coordinates of the vector u(ej ) in the basis
f , i.e. the scalars which verify:
q
ÿ
u(ej ) = ⁄ij f i . (2.5)
i=1

We denote Mate,f (u) or [u]fe the1 p ◊ q matrix (⁄ij )16i6q,16j6p . It is called the matrix of the linear
map u from basis e to f .
It is clear that the columns of the matrix [u]fe are the column matrices [u(ej )]f . In other words:
The matrix of a linear map u : E æ F , from basis e to f , has, as columns, the column matrices of the
coordinates of the image of the vectors of basis e of E, in the basis f of F :
Ä ä
[u]fe = [u(e1 )]f · · · [u(ep )]f .

Example 5.2.1 (Matrix representation of a linear map)

Let u : R3 æ R2 be de�ned, for all (x, y, z) in R3 , by setting:

u(x, y, z) := (x + y ≠ 4z, ≠3x + 2y + 5z).

De�ne B := (e1 , e2 , e3 ) the standard basis of R3 and BÕ := (f 1 , f 2 ) the standard basis of R2 .

Denote A the matrix of the linear map u from basis e to f . We easily compute

u(e1 ) = (1, ≠3) = 1f 1 ≠3f 2 u(e2 ) = (1, 2) = 1f 1 + 2f 2 u(e3 ) = (≠4, 5) = ≠4f 1 + 5f 2 .

Thus we can write:

u(e1 ) u(e2 ) u(e3 )
Ç å
A = [u]fe = 1 1 ≠4 f1
≠3 2 5 f2

It is clear that, if u and v are both linear maps in L(E, F ), we have, for all scalar ⁄,
[⁄u + v]fe = ⁄ [u]fe + [v]fe

Remark 5.2.2
Note that one can also refer to [u]fe as Mate,f (u).

Fondamental Formulae
We will see in this paragraph how the notations we introduced in the previous paragraph are adapted
to the operations already de�ned, from one hand on Matrices, and, from the other hand on linear maps.
Let E be a �nite-dimensional vector space with dim(E) = p, F be a �nite-dimensional vector space
with dim(F ) = q and u : E æ F be a linear map. With the notations introduced at (2.4) and (2.5), we
easily see that, for all vector x in E,
p
! p
ÿ ÿ
u(x) = u –j ej = –j u(ej )
j=1 j=1
p q q p
!
ÿ ÿ ÿ ÿ
= –j ⁄ij f i = ⁄ij –j f i.
j=1 i=1 i=1 j=1
1
The notation [u]fe comes from [CPY96, Chap. III]

72 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

This latter formula can be naturally summarized by:

[u(x)]f = [u]fe · [x]e . (2.6)

Conversely, if A is a matrix in Mp,q (K), the map u : X ‘æ AX, from Mp,1 (K) to Mq,1 (K) is a linear
map. Ifone identi�es Mp,1 (K) with Kp and Mq,1 (K) with Kq using standard basis e of Kp and f of
Kq , we see that, for all vector X of Kp , we have the equality:

[u]fe X = [u]fe [X]f = [u(X)]f = u(X) = AX.

We deduce that [u]fe = A.

Let E be a �nite-dimensional vector space with dim(E) = p, F be a �nite-dimensional vector space
with dim(F ) = q and G be a �nite-dimensional vector space with dim(G) = r. Let e, f and g be
basis of E, F and G and let u : E æ F and v : F æ G be two linear maps. We see that, for all vector
X of E, we have the equality:
[v ¶ u]ge = [v]gf [u]fe . (2.7)
One can also check directly the previous formula by writting:
Ç
q å q q r q Ç r
å
ÿ ÿ ÿ ÿ ÿ ÿ
v ¶ u(ej ) = v ⁄ij f i = ⁄ij v (f i ) = ⁄ij µki g k = ⁄ij µki gk .
i=1 i=1 i=1 k=1 i=1 k=1

A Particular kind of linear maps

Endomorphisms
Let E be a �nite-dimensional vector space with dim(E) = p, e be a basis of E and u be an endomor-
phism of E. Since both the domain and codomain are the same, it is natural (but not mandatory) to
take the same basis. The matrix [u]ee is called the matrix of the endomorphism u in the basis e.
hence, whatever the basis e of E, we have IdE (ei ) = ei . Therefore, [IdE ]ee = Ip . Besides, Applying
Formula (2.7), we get [u ¶ u]ee = [u]ee · [u]ee = ([u]ee )2 . Thus, by induction, we get

[un ]ee = ([u]ee )n , ’n œ N. (2.8)

If u is bijective, then this formula holds for every n in Z.

Proposition 5.2.3
Let E and F be two �nite-dimensional vector spaces with dim(E) = p and dim(F ) = q. Let
e := (e1 , · · · , ep ) be a basis of E, f := (f 1 , · · · , f p ) be a basis of F and u : E æ F be a linear
map. The map u is bijective if, and only if, p = q and if the matrix [u]fe is invertible. In this latter
⇥ ⇤e Ä ä≠1
case, we have the equality: u≠1 f = [u]fe .

Proof 5.2.4
To be �lled! ⇤

Rank of a Linear Map

73 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Theorem 5.2.5
Let E and F be two �nite-dimensional vector spaces, e being a basis of E and f being a basis of F
and let u : E æ F be a linear map. The rank of the matrix [u]fe equals the rank of the linear map u.
In particular it does not depend on the choice of the basis e nor f .

Proof 5.2.6
To be �lled To be �lled! ⇤

Corollary 5.2.7
Let q be a non-zero integer, S := (x1 , · · · , xq ) be a family of vectors of a �nite-dimensional vector
space E and e be a basis of E. The rank of S (which is by de�nition the rank of vector subspace
Span(S) equals the rank of the matrix

([x1 ]e · · · [xq ]e ) .

Proof 5.2.8
To be �lled To be �lled! ⇤

5.3 Change of Basis

Notations
Let E be a �nite-dimensional vector space with dim(E) = p and let e and f be two basis of E. In
order to make a clear distinction between these two basis, we will use the following terminology.

Change of Basis Matrix

The change of basis matrix, denoted P , from the former basis e to the new one f (or simply the
change of basis matrix P from e to f ) is the matrix the columns of which represent the vectors of
the new basis, given in the coordinate system of the former basis.
⇥ ⇤e
P = [IE ]ef = [f 1 ]e · · · f p . (3.9)

Example 5.3.1 (Change of Basis Matrix)

Let e := (e1 , e2 , e3 ) be the standard basis of R3 and de�ne
8
< f 1 := 2e2 + 3e3
>
f 2 := 2e1 ≠ 5e2 ≠ 8e3
>
:
f 3 := ≠e1 + 4e2 + 6e3 .
De�ne f := (f 1 , f 2 , f 3 ). One can easily verify that f is a linearly independent family of vectors
and therefore a basis of R3 . Denote P the change of basis matrix from e to f . According to (3.9), we
can write that: Ö è
0 2 ≠1
P := [IE ]ef := 2 ≠5 4 .
3 ≠8 6

74 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Proposition 5.3.2
The change of basis matrix P from the basis e to f . Its inverse P ≠1 is the change of basis matrix
from the basis f to e.

Proof 5.3.3
To be �lled! ⇤

Example 5.3.4 (Change of Basis Matrix: continuation)

We keep the notations from Example 5.3.1. Since P the change of basis matrix from e to f , we know
that P ≠1 will be the change of basis matrix from f to e.. According to (3.9), we can write that:
Ö è
0 2 ≠1
P := [IE ]ef := 2 ≠5 4 .
3 ≠8 6

and then compute P ≠1 . One then gets: We easily get:

Ö è
2 ≠4 3
[IE ]fe = P ≠1 = 0 3 ≠2 .
≠1 6 ≠4

De�nition 5.3.5 (Rank)

Let F be a �nite family of vectors of E. The rank of F, denoted rk(F) or rk F, is the dimension of
the vector subspace of E spanned by F.

The case of vectors

The components of a vector x are modi�ed by a change of basis (from e to f let’s say). Precisely the
relation is:

[x]f = [IE (x)]f = [IE ]fe · [x]e .

One can translate this equality by writing:

Matrix of change of basis

Let X be the column matrix of the components of a vector in the former basis, Y be the column
matrix of the components of the same vector in the new basis and let P denote the change of basis
matrix. We then have the following equality:

Y = P ≠1 X. (3.10)

75 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Example 5.3.6
We keep the data given at Example 5.3.1. Denote x the vector de�ned by x := 2e1 ≠ 3e2 + 4e3 .
Denote also X := [x]e and Y := [x]f . It is clear that
Ö è
2
[x]e = ≠3 .
4

According to (3.10), it is easy to determine the components of the vector x in the basis f . We hence
can write:
Ö è Ö è Ö è
2 ≠4 3 2 28
Y = [x]f = [IE (x)]f = [IE ]fe · [x]e = P ≠1 · X = 0 3 ≠2 · ≠3 = ≠17 .
≠1 6 ≠4 4 ≠36

The case of Linear Maps

Let E and F be two �nite-dimensional vector spaces with dim(E) = p and dim(F ) = q. Let e and eÕ
be two basis of E, f and f Õ be two basis of F and let u : E æ F be a linear map. The obvious equality
u = IF ¶ IE can be translated by:
Õ Õ
[u]feÕ = [IF ]ff · [u]fe · [IE ]eeÕ . (3.11)

Remark 5.3.7 ("Segment Addition Postulate")

There is a simpli�cation one can perform, which allows to remember the way Formula (3.11) works.
It is a kind of "Segment Addition Postulate". Indeed, (3.11) can be read as:
Õ Õ
[IF ]ff · [u]fe · [IE ]eeÕ = [u]feÕ .

Every bottom left map cancels with the same map, at the top right. It reminds what happens, in a
di�erent order (i.e. every number at the top right cancels with the bottom left number), with the
Riemann integral of an integrable function over an interval [a, b]. For every c in [a, b]:
⁄ c ⁄ b ⁄ c
⇤
f= f+ f,
a a b⇤

In order to remember Formula (3.11), we introduce below standard conventions (that must be respected
if one does not want to make mistakes). Basis e and f are said to be “former basis” while eÕ and f Õ are
said to be “new basis”.

Let u : E æ F be a linear map. Let e to eÕ be two basis of E and f and f Õ be two basis of F . Let A
be the matrix representation of u from the former basis e and f and let B be the matrix of u from
the new basis eÕ to f Õ .
Denote P the change of basis matrix from e to eÕ and denote Q the change of basis matrix from f
to f Õ . We have the following equality:

B = Q≠1 AP. (3.12)

76 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Example 5.3.8
We keep the linear map de�ned at Example 5.2.1. Hence u : R3 æ R2 , is de�ned, for all (x, y, z) in
R3 , by:
u(x, y, z) := (x + y ≠ 4z, ≠3x + 2y + 5z).
e := (e1 , e2 , e3 ) denotes the standard basis of R3 while f := (f 1 , f 2 ) denotes the standard basis
of R2 . De�ne eÕ := (eÕ1 , eÕ2 , eÕ3 ) and f Õ := (f Õ1 , f Õ2 ) by:
8
< e1 := 2e2 + 3e3 ® Õ
> Õ
f 1 := 2f 1 ≠ 3f 2
e2 := 2e1 ≠ 5e2 ≠ 8e3
Õ &
>
: Õ f Õ2 := 3f 1 ≠ 4f 2
e3 := ≠e1 + 4e2 + 6e3

Let A be the matrix representation of u from the former basis e and f . From Example 5.2.1, we know
that Ç å
1 1 ≠4
A= .
≠3 2 5
We have seen, in Example 5.3.1, that eÕ is a basis of R3 . For the same reason, f Õ is a basis of R2 . If
P denotes the change of basis matrix from e to eÕ and Q the change of basis matrix from eÕ to f Õ ,
then one can write:
Ö è
0 2 ≠1 Ç å
2 3
P = 2 ≠5 4 & Q= .
≠3 ≠4
3 ≠8 6

Let B be the matrix representation of u from the new basis eÕ and f Õ . We have the equality:
Ö è
Ç åÇ å 0 2 ≠1 Ç å
≠4 ≠3 1 1 ≠4 ≠17 52 ≠39
B = Q≠1 AP = 2 ≠5 4 = .
3 2 ≠3 2 5 8 ≠25 19
3 ≠8 6

Proposition 5.3.9
Let A be a square matrix of size p and of rank r. There exist two square invertible matrices, denoted
G and D such that GAD = Jr , where we de�ned:
0 1
1 0 ··· 0
B .. C
B0
B . C
C
B C
B 1 C Ω row r
B. .. C
Jr =B
B ..
C
0 .C
B C
B .. C
@ . A
0 ··· 0

Proof 5.3.10
To be �lled To be �lled! See [CPY96, p.85 Proposition III.5] ⇤

77 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Remark 5.3.11
1. Note that Jr can also be denoted by
Ç å
Ir 0p≠r,p≠r
(3.13)
0p≠r,p≠r 0r,r

where 0s,t denotes the null matrix which has s rows and t columns.

2. Multiply a matrix A, from the right hand side (respectively from the left hand side) by a square
and invertible matrix comes to make a change of basis in the departure space (respectively
to make a change of basis in the arrival space) of the map u. It does not modify or a�ect
the rank of the matrix: since A and B both represent the linear map u, we have the equality
rk(A) = rk u = rk(B). Since the rank of Jr is obviously r, we see that the property given in
Proposition 6.3.2 characterizes the matrices with rank r.

3. The way to obtain matrices G and D is to perform Gaussian Elimination (see Chapter 1 for
more details).

De�nition 5.3.12 (Rank of a linear map)

Let E and F be two K vector spaces and u : E æ F be a linear map. We call rank of u, and we
denote it rk(u), the dimension of Im(u), when it is �nite.

Corollary 5.3.13
The rank of a matrix equals the rank of its transpose.

Proof 5.3.14
To be �lled To be �lled! See [CPY96, p.86 Corollaire III.7] ⇤

Corollary 5.3.15
The rank of a p ◊ q matrix equals the rank of the family of its rows vectors in Kq .

Proof 5.3.16
To be �lled! See [CPY96, p.86 Corollaire III.8] ⇤

De�nition 5.3.17 (Equivalent Matrices)

Two matrices A and B of Mp,q (K) are said to be equivalent when it exists a q ◊ q invertible matrix,
denoted P , and a p ◊ p invertible matrix, denoted Q such that:

B = QAP.

Remark 5.3.18
It results from the previous de�nition that all matrices which represent a given linear map u are
equivalent to each other.

78 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Example 5.3.19
Matrices A and B, from Example 5.3.8 are equivalent.

The case of Endomorphisms

In the case where the linear map u is an endomorphism of a vector space E, there is, obviously, only
one former basis, and only one new basis. The rule therefore becomes:

Let A be the matrix of an endomorphism u : E æ E in the former basis and let B be the matrix of
the endomorphism u in the new basis. Denote P the change of basis matrix. We have the following
equality:
B = P ≠1 AP. (3.14)

De�nition 5.3.1 (Eigenvalue Space). Let u be an endomorphism of a K vector space E and let ⁄
be an eigenvalue of u. The vector subspace E⁄ := ker(u ≠ ⁄IE ), i.e. the set of all vectors x of E such
that u(x) = ⁄ x, is called the eigenspace of u associated with the eigenvalue ⁄.

Example 5.3.20
Let B := (e1 , e2 , e3 ) be the standard basis of R3 and let u : R3 æ R3 be the linear map de�ned by
setting: 8
< u(e1 ) := ≠e1 + 2e2 + 2e3
>
u(e2 ) := 2e1 ≠ e2 + 2e3
>
:
u(e3 ) := 2e1 + 2e2 ≠ e3 .
De�ne E≠3 := {t œ R3 , f (t) = ≠3t} and E3 := {t œ R3 , f (t) = 3t}. De�ne f 1 := e1 ≠ e3 ,
f 2 := e2 ≠ e3 and f 3 := e1 + e2 + e3 . Denote f := (f 1 , f 2 , f 3 ).
1. Determine the matrix representation of u, from basis e and e, which will be denoted M .
2. Show that f is a basis of R3 .
3. Determine the matrix representation of u, from basis f to f , which will be denoted D.
4. Compute M n , for all n in Nú .
Answers
1. It is clear that the we have the equality:
Ö è
≠1 2 2
M := [u]ee = Mate (u) = 2 ≠1 2 .
2 2 ≠1
2. One just has to solve, in ⁄1 , ⁄2 and ⁄3 , the following linear system:
Ö è Ö è Ö è Ö è
≠1 2 2 0
⁄1 2 + ⁄2 ≠1 + ⁄3 2 = 0 ,
2 2 ≠1 0

79 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

the solution of which is clearly ⁄1 = ⁄2 = ⁄2 = 0. This proves that the vectors which
constitues the family of vectors f are linearly independent.

3. Denote P the change of basis matrix from e to f , one can write:

Ö è
1 0 1
P := [I E ]ef = 0 1 1 .
≠1 ≠1 1

One easily sees that: Ö è

2
3 ≠ 13 ≠ 13
P ≠1 = [I E ]fe = ≠ 13 2
3 ≠ 13 .
1 1 1
3 3 3

We can then use (3.14) to write that:

D := [u]ff = [I E ]fe · [u]ee · [I E ]ef .

In other words we have:

Ö 2
è Ö è Ö è
3 ≠ 13 ≠ 13 ≠1 2 2 1 0 1
D = Matf (u) = P ≠1 · M · P = ≠ 13 2
3 ≠ 13 · 2 ≠1 2 · 0 1 1
1 1 1
3 3 3 2 2 ≠1 ≠1 ≠1 1
Ö è
≠3 0 0
= 0 ≠3 0 .
0 0 3

4. The equality
D = P ≠1 M P (3.15)
can also be read (by multiplying (3.15) by P from the left-hand side and by P ≠1 from the right-
hand side):
M = P DP ≠1
Therefore, one can write:

M n = P DP ≠1 P DP ≠1 · · · P DP ≠1 = P Dn P ≠1 , (3.16)

and thus
Ö èÖ èÖ 2 è
1 0 1 (≠3)n 0 0 3 ≠ 13 ≠ 13
M =
n
0 1 1 0 (≠3) n
0 ≠ 13 2
3 ≠ 13
1 1 1
≠1 ≠1 1 0 0 3n 3 3 3
Ü ê
2(≠3)n 3n (≠3)n 3n (≠3)n 3n
3 + 3 ≠ 3 + 3 ≠ 3 + 3
(≠3)n 2(≠3)n
= + 33 ≠ (≠3)
n
≠ 3 + 33 + 33
n n n
3 3
.
≠ (≠3) ≠ (≠3) 2(≠3)n
n n
3n 3n 3n
3 + 3 3 + 3 3 + 3

80 S����� 2023
L����� A������ Lecture Notes C������ 5: Use of Basis

Remark 5.3.21
1. In Example 5.3.20, computing An , without using the relation between A and D would be im-
possible.

2. We will see, in Chapter 7, a systematic way to determine the vectors of the family f (of Example
5.3.20), when they exist.

As we will see in Chapter 7, a basis f , as we had in Example 5.3.20, does not necessarily exist. However,
it is still possible to give the matrix representation of a linear map between any basis we choose.

De�nition 5.3.22 (Similar Matrices)

Two matrices A and B of Mp (K) are said to be similar when there exists a p ◊ p invertible matrix
P such that:
B = P ≠1 AP.

Remark 5.3.23
It results from the previous de�nition that all matrices which represent the same given endomor-
phism u are similar to each other.

Example 5.3.24
Matrices A and D, from Example 5.3.20, are similar.

81 S����� 2023
Part III

82
C������ 6

Multilinear Maps & Determinants

Ce qu’a fait Gourdon est 100 fois mieux. Regarder comment modi�er ce chapitre de A à Z en prenant
Exemple sur [Gou09, Chap. III.4]
Rajouter des exemples pour chaque section car il n’y en n’a pas
un seul!
In this entire chapter p denotes a positive integer.

6.1 De�nitions
They are many di�erent ways to de�ne the notion of Determinant. Of course they are all equivalent.
In this course we will start from p-linear function to do so.
We identify in the sequel a vector x of Kp with the column matrix [x]e , where e denotes the standard
basis of Kp (in other words one write the components of the vector x in column). In particular it is
the same thing to consider a family (x1 , x2 , · · · , xp ) of p vectors of Kp or to consider the p ◊ p matrix
A := ([x1 ]e · · · [xp ]e ), the columns of which are the column matrices [xj ]e .

De�nition 6.1.1 (Alternating p-linear form)

An Alternating p-linear form on Kp is a map b : Mp (K) æ K such that:

1. Whatever the index i, when one freezes all the columns of a matrix
A := ([x1 ]e · · · [xi ]e · · · [xp ]e ), except for the ith , one gets a map Xi ‘æ
b ([x1 ] · · · [xi≠1 ] Xi [xi+1 ] · · · [xp ] ) from K to K which is linear.
e e e e p

2. If the matrix A has two equal columns (Xi = Xj for two di�erent indices i and j) then b(A) =
0.

Proposition 6.1.2
Let b be an alternating p-linear form on Kp and B be the matrix obtained by exchanging two columns
of Matrix A. One has the equality b(B) = ≠b(A).

Proof 6.1.3
To be �lled To be �lled! See [CPY96, p.106 Proposition IV.13] ⇤

83
L����� A������ Lecture Notes C������ 6: A First Look at Determinants

Proposition 6.1.4
An alternating p-linear form on Kp is completely determined by the value of b(Ip ), where Ip denotes
the identity matrix of size p.

Proof 6.1.5
To be �lled To be �lled! See [CPY96, p.106 Proposition IV.14] ⇤

Remark 6.1.6
It is easy to verify that the set B of the alternating p-linear forms on Kp is a vector subspace of the
vector space KMp (K) . The previous proposition shows that two alternating p-linear forms on Kp are
collinear. In other words, the vector space B has a dimension not greater than 1. In order to show
that dim(B) is 1 exactly, one just needs to �nd an alternating p-linear forms on Kp di�erent from 0.

Let b be the alternating p-linear form de�ned on Mp (K) by setting

ÿ
b(A) := sgn(‡) a‡(1),1 a‡(2),2 · · · a‡(p),p · b(Ip ), (1.1)
‡œ ‡p

where A := (ai,j )(i,j)œJ1,pK belongs to Mp (K), ‡p denotes the set of all permutations 1 of {1, · · · , p}
and sgn(‡) denotes the signature of the permutation ‡. In (1.1), the sum is computed over all permu-
tations ‡ of the set {1, 2, ..., p}.

Remark 6.1.7
Note that (1.1) also reads
ÿ p
Ÿ
b(A) := sgn(‡) a‡(i),i · b(Ip ). (1.2)
‡œ ‡p i=1

One can check that the form b, de�ned in (1.1), for which we impose that b(Ip ) := 1, is a non zero
alternating linear form. This is this particular linear form we will be interested in in the sequel.

De�nition 6.1.8 (Determinant)

We call determinant, and we denote det, the alternating p-linear form on Kp such that det(Ip ) = 1.

Remark 6.1.9
1. Gathering both Equality (1.2) and De�nition 6.1.8 we obtain the Leibniz formula, which reads
for an p ◊ p matrix A:
ÿ p
Ÿ
det(A) := sgn(‡) a‡(i),i . (1.3)
‡œ ‡p i=1

2. Note that the determinant is only de�ned for square matrices.

1
A permutation is a function that reorders this set of integers (here the set is 1, 2, ..., p). The value in the ith position after
the reordering ‡ is denoted by ‡(i). For example, for p = 3, the original sequence 1, 2, 3 might be reordered to ‡ := (2, 3, 1)
(i.e. ‡(1) = 2, ‡(2) = 3 and ‡(3) = 1). The set of all such permutations (also known as the symmetric group on p elements)
is denoted by ‡p . For each permutation ‡, sgn(‡) denotes the signature of ‡, a value that is +1 whenever the reordering
given by ‡ can be achieved by successively interchanging two entries an even number of times, and ≠1 whenever it can be
achieved by an odd number of such interchanges.

84 S����� 2023
L����� A������ Lecture Notes C������ 6: A First Look at Determinants

3. We sometimes use the notation

a1,1 a1,2 · · · a1,q

a2,1 a2,2 · · · a2,q
.. .. ..
. . .
ap,1 ap,2 · · · ap,q

to denote det(A).

6.2 Properties of Determinants

Theorem 6.2.1
Determinant have the following properties.

1. A determinant of a matrix with contains two identical columns is zero.

2. A determinant of a matrix which has a column full of zeros.

3. When one changes two columns of a determinant the sign changes.

4. The value of a determinant will not change if one adds, to one of its columns a linear combi-
nation of its other columns.

5. If the family of column matrices A1 , · · · , Ap is not linearly independent (i.e. the corresponding
family of vectors of Kp is not linearly independent) then det(A1 , · · · Ap ) = 0.

Proof 6.2.2
To be �lled! See [CPY96, p.108 Proposition IV.17] ⇤

A direct consequence of the p-linearity of the determinant is that, for all ⁄ in K and A in Mp (K), we
have:
det(⁄A) = ⁄p det(A). (2.4)

Theorem 6.2.3
Let A and B be two matrices in Mp (K). We have the equality:

det(AB) = det(A) · det(B). (2.5)

Proof 6.2.4
To be �lled! See [CPY96, p.109 Theorème IV.18] ⇤

Theorem 6.2.5
In order the p ◊ p matrix A is invertible it is necessary and su�cient that its determinant is non zero.
In this latter case, one has the equality:
1
det(A≠1 ) = . (2.6)
det(A)

85 S����� 2023
L����� A������ Lecture Notes C������ 6: A First Look at Determinants

Proof 6.2.6
To be �lled! See [CPY96, p.109 Theorème IV.19] ⇤

Theorem 6.2.7
For every matrix A of Mp (K), the following equality holds:

det(tA) = det(A).

Proof 6.2.8
To be �lled To be �lled! See [CPY96, p.109 Theorème IV.19] ⇤

Theorem 6.2.9
Determinant have the following properties.

1. A determinant of a matrix with contains two identical rows is zero.

2. A determinant of a matrix which has a row full of zeros.

3. When one exchanges two rows of a determinant the sign changes.

4. The value of a determinant will not change if one adds, to one of its rows a linear combination
of its other rows.

5. If the rows of matrices of a square matrix A are not linearly independent (i.e. the corresponding
family of vectors of Kp is not linearly independent) then det(A) = 0.

Proof 6.2.10
To be �lled One applies Theorem to the matrix tA since the columns of this latter are nothing but the
rows of A. ⇤

6.3 Expanding a Determinant Along a Row or a Column

De�nition 6.3.1
Let A := (aij ) be a matrix of Mp (K). We call minor of the coe�cient aij in A, and we denote it
Mij (A), the determinant of the matrix of Mp≠1 (K) obtained by removing the ith row and the j th
column of A.

Proposition 6.3.2
We have the equality:
p
ÿ
det(A) = aij (≠1)i+j Mij (A).
j=1

Proof 6.3.3
To be �lled! See [CPY96, p.110 Proposition IV.23]

86 S����� 2023
L����� A������ Lecture Notes C������ 6: A First Look at Determinants

De�nition 6.3.4 (Cofactor)

We call cofactor of the coe�cient aij in A, the scalar de�ned by:

Cij (A) = (≠1)i+j Mij (A).

Remark 6.3.5 (Rules concerning signs)

The coe�cient (≠1)i+j equals 1 or ≠1.

Theorem 6.3.6
Let A := (aij )16i,j6p be a matrix of Mp (K). We have the following equalities:
p
ÿ
det(A) = aij Cij (A) (expanding along the ith row),
j=1
ÿp
det(A) = aij Cij (A) (expanding along the j th column).
i=1

Proof 6.3.7
To be �lled To be �lled! See [CPY96, p.112 Theoreme IV.25] ⇤

Corollary 6.3.8
If det(A) ”= 0, then
1
A≠1 = t
(Ã)
det(A)
where à is the matrix of cofactors i.e. Ãij = Cij (A), for every (i, j) in J1, pK.

Proof 6.3.9
To be �lled To be �lled! See [CPY96, p.112 Corollaire IV.26] ⇤

6.4 Computing a Determinant in practice

• Determinant of a triangular matrix
If A := (aij )16i,j6p is a lower triangular matrix of Mp (K), by expanding, successively along
the �rs row, one gets:

a1,1 0 ··· 0 a2,2 0 ··· 0

.. .. .. .
.. .
.. .. .. ..
. . . . . . = a ···a .
= a1,1 1,1 p,p
ap,1 · · · ap≠1,p≠1 0 ap≠1,2 · · · ap≠1,p≠1 0
ap,1 · · · ap,p≠1 ap,p ap,2 · · · ap,p≠1 ap,p

In view of Theorem ??, it is clear that the determinant of an upper triangular matrix equals the
product of its diagonal terms.

87 S����� 2023
L����� A������ Lecture Notes C������ 6: A First Look at Determinants

• Make zeros appear

When the matrix is not triangular, “one can make appear zeros” by adding to a row (or a column)
a linear combination of the others. It will not modify the value of the determinant. One can also
move these zeros by exchanging two rows or two columns; of course this will modify the sign
of the determinant. One can also multiply a row or a column by a scalar, which multiplies the
determinant by this very scalar.
All these operations remind the elementary operations of the Gaussian Elimination. The dif-
ference here is that, for the computation of a determinant one can work on rows as well as on
columns as the same time.
By using some of the elementary transformations described above one can reduce the computa-
tion of a determinant to the case where the matrix is triangular. This will therefore achieves the
computation in view of the �rst point above.

• Expansion
The general method (i.e. the one described above) gives good results for numerical matrices (i.e.
matrices with numbers instead of letters). However one can often improve it using the expansion
along a row or a column when not many non zero coe�cients left in this very row or column.
When one wants to compute a determinant, denoted let’s say p which has a particular form
(with p rows and p columns and p being unspeci�ed), one can use this way of computing deter-
minant in order to get a recursive formula which links p and p≠1 .

• Numerical Techniques
In order to compute a determinant using a computer, we use methods adapted to each speci�c
case. The LU factorization is a good example of such a method.

De�nition 6.4.1 (Extracted Determinant)

Let A := (aij ) be a matrix of Mp,q (K). We call extracted determinant (of A) of order r 6 min{p, q}
the determinant of a square matrix of order r “extracted” from A. In other words by canceling p ≠ r
rows and q ≠ r columns in the matrix A.

Corollary 6.4.2
The rank of a matrix is the greatest order of an extracted determinant di�erent from 0.

Proof 6.4.3
To be �lled To be �lled! See [CPY96, p.114 Corollaire IV.30] ⇤

Theorem 6.4.4
The rank of a matrix equals the rank of its transpose.

Proof 6.4.5
To be �lled! See [CPY96, p.115 Corollary IV.31] ⇤

88 S����� 2023
Part IV

89
C������ 7

Diagonalization

7.1 Eigenvalues of an Endomorphism

De�nitions

De�nition 7.1.1 (Eigenvalue & Eigenvector)

Let u be an endomorphism of a K vector space E. We say that a scalar ⁄ is an eigenvalue of u if
there exists a non zero vector x of E such that:

u(x) = ⁄ x.

Such a vector x is called an eigenvector of u associated to the eigenvalue ⁄.

Remark 7.1.2
Since, whatever the scalar ⁄, we have u(0) = 0 = ⁄ 0, the requirement x ”= 0 is essential in the
previous de�nition.

De�nition 7.1.3 (Eigenvalue Space)

Let u be an endomorphism of a K vector space E and let ⁄ be an eigenvalue of u. The vector subspace
E⁄ := ker(u ≠ ⁄IE ), i.e. the set of all vectors x of E such that u(x) = ⁄ x, is called the eigenspace
of u associated with the eigenvalue ⁄.

Example 7.1.4 (Eigen values and eigen vectors)

Let B := (e1 , e2 , e3 ) be the standard basis of R3 and let f : R3 æ R3 be a linear map de�ned by
setting: 8
2 2
< f (e1 ) := 3 e1 + 3 e2 + 3 e3
≠1
>
f (e2 ) := 23 e1 + ≠1 2
3 e2 + 3 e3
>
:
f (e3 ) := 23 e1 + 23 e2 + ≠1
3 e3 .

De�ne E≠1 := {u œ R3 , f (u) = ≠u} and E1 := {u œ R3 , f (u) = u}.

1. Prove that e1 ≠ e2 and e1 ≠ e3 both belong to E≠1 while e1 + e2 + e3 belongs to E1 .

2. Deduce that e1 ≠ e2 , e1 ≠ e3 and e1 + e2 + e3 are eigenvalues of u.

90
L����� A������ Lecture Notes C������ 7: Diagonalization

1. De�ne u := e1 ≠ e2 , v := e1 ≠ e3 and w := e1 + e2 + e3 . One can obviously write:

≠1 2 2 2 ≠1 2
f (u) = f (e1 ≠ e2 ) = f (e1 ) ≠ f (e2 ) = e1 + e2 + e3 ≠ ( e1 + e2 + e3 ) = ≠e1 + e2
3 3 3 3 3 3
= ≠u.
and
≠1 2 2 2 2 ≠1
f (v) = f (e1 ≠ e3 ) = f (e1 ) ≠ f (e2 ) = e1 + e2 + e3 ≠ ( e1 + e2 + e3 ) = ≠e1 + e3
3 3 3 3 3 3
= ≠v.

These two equalities prove that both u and v belong to

E≠1 .
Moreover, we have:
f (e1 + e2 + e3 ) = f (e1 ) + f (e2 ) + f (e3 )
≠1 2 2 2 ≠1 2 2 2 ≠1
= e1 + e2 + e3 + e1 + e2 + e3 + e1 + e2 + e3
3 3 3 3 3 3 3 3 3
= e1 + e2 + e3 .

Thus w belongs to E1 .

2. From the previous question we deduce that both u and v are eigenvectors associated to the
eigenvalue ≠1. Moreover E≠1 is the eigenspace associated to the eigenvalue ≠1.
Besides, and that w is an eigenvector associated to the eigenvalue 1. and E1 is the eigenspace
associated to the eigenvalue 1.

Proposition 7.1.5
Let E be a �nite-dimensional K-vector space with dim(E) = p and let u be an endomorphism of
E. The scalar det([u]ee ) does not depend on the basis e of E. We denote it det(u) and is called the
determinant of the endomorphism u.

Proof 7.1.6
To be �lled To be �lled! See [CPY96, p.136 Proposition V.3] ⇤

Proposition 7.1.7
Let E be a K-�nite-dimensional vector space with dim(E) = p, u an endomorphism of E and ⁄ be
an element of K. The following conditions are equivalent.

1. ⁄ is an eigenvalue of u.

2. the endomorphism u ≠ ⁄IE is not injective.

3. det(u ≠ ⁄IE )=0.

Proof 7.1.8
To be �lled To be �lled! See [CPY96, p.136 Proposition V.4] ⇤

91 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

De�nition 7.1.9 (Characteristic Polynomial)

Let E be a K-�nite-dimensional vector space with dim(E) = p and u be an endomorphism of E.
Let us choose a basis e of E. It is easy to see that the quantity:

‰u (X) = det(u ≠ XIE ) = det([u ≠ XIE ]ee ) = det([u]ee ≠ XIE )

is a polynomial in X. Its degree equals p = dim(E). This polynomial is called the Characteristic
polynomial.

Applications to the Calculation of Eigenvalues

One computes �rst the characteristic polynomial ‰u , then one choose a basis e of E and one computes
the determinant of the matrix [u]ee ≠ XIE . The eigenvalues of u are the roots, in K, of the polynomial
‰u .

Example 7.1.10 (Characteristic Polynomials)

Let us keep the notations of Example 7.1.4. One can write:

‰u (X) = det([u]ee ≠ XIE ) = det(A ≠ XI3 ),

Ö ≠1 2 2
è Ö è
3 3 3 ≠1 2 2
2 2 1
where we have set A := ≠1 . Since A = 2 ≠1 2 , one can write:
3
2
3
2
3 3
3 3
≠1
3 2 2 ≠1

≠1 ≠ 3X 2 2
1
‰u (X) = det( · (3A ≠ 3XI3 )) = 3≠3 det(·(3A ≠ 3XI3 )) = 2 ≠1 ≠ 3X 2
3
2 2 ≠1 ≠ 3X
Å ã3 3 ≠ 3X 2 2 1 2 2
1 (3 ≠ 3X)
= 3 ≠ 3X ≠1 ≠ 3X 2 = 1 ≠1 ≠ 3X 2
3 33
3 ≠ 3X 2 ≠1 ≠ 3X 1 2 ≠1 ≠ 3X
1 2 2
(3 ≠ 3X) (3 ≠ 3X)
= 0 ≠3 ≠ 3X 0 = · (3 + 3X)2 = (1 ≠ X) · (1 + X)2 .
33 33
0 0 ≠3 ≠ 3X

7.2 Diagonalizable Endomorphisms

De�nition 7.2.1 (Diagonalizable Endomorphism)

An endomorphism u, of a �nite dimensional vector space E, is said to be diagonalizable if there exists
a basis f of E such that the matrix [u]ff is diagonal.
Diagonalize an endomorphism u means �nding a basis f of E for which the matrix [u]ff is diagonal.
Of course this is possible only if the endomorphism u is diagonalizable.

92 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

Proposition 7.2.2
Let E be a K-�nite-dimensional vector space with dim(E) = p, u be an endomorphism of E and
let f := (f 1 , · · · , f p ) be a basis of E. The matrix [u]ff is diagonal if, and only if, the vectors f i
are eigenvectors of u. The ith coe�cient of its diagonal is then the eigenvalue of u associated to the
corresponding eigenvector f i .

Proof 7.2.3
To be �lled To be �lled! See [CPY96, p.138 Proposition V.8] ⇤

Proposition 7.2.4
If an endomorphism u of a K-�nite-dimensional vector space E, with dim(E) = p is diagonalizable,
its characteristic polynomial ‰u splits over K (there are p roots of ‰u in K. Each of these roots is
counted which its multiplicity).

Proof 7.2.5
To be �lled To be �lled! See [CPY96, p.139 Proposition V.9] ⇤

Lemma 7.2.6
Let u be an endomorphism of a K-�nite-dimensional vector space E, with dim(E) = p and
⁄1 , · · · , ⁄m some distinct eigenvalues of u and x1 , x2 , · · · , xm eigenvectors of u associated to
⁄1 , · · · , ⁄m such that u(xi ) = ⁄i xi (i.e. xi œ ker(u ≠ ⁄i IE ).
If x1 + · · · + xm = 0 then x1 = · · · = xm = 0.

Proof 7.2.7
To be �lled To be �lled! See [CPY96, p.139 Proposition V.10] ⇤

Remark 7.2.8
The previous result means that, for distinct eigenvalues ⁄1 , · · · , ⁄m , the corresponding eigenspaces
E⁄i := ker(u ≠ ⁄i IE ) all are in direct sum.

Theorem 7.2.9
Let u be an endomorphism of a K-�nite-dimensional vector space E, with dim(E) = p. If the char-
acteristic polynomial of u has p distinct roots in K, i.e. if u has p eigenvalues, then u is diagonalizable.

Proof 7.2.10
To be �lled! See [CPY96, p.140 Theorème V.12]
⇤

Proposition 7.2.11
Let u be an endomorphism of a K-�nite-dimensional vector space E and let ⁄ be an eigenvalue of
u. Denote r the multiplicity order of ⁄ as a root of ‰u . We have the following inequality:

dim(ker(u ≠ ⁄IE )) 6 r.

93 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

Proof 7.2.12
To be �lled! See [CPY96, p.140 Proposition V.13] ⇤

Proposition 7.2.13 (Diagonalizability Criteria for an endomorphism)

Let u be an endomorphism of a K-�nite-dimensional vector space E with dim(E) = p. Denote
{⁄1 , · · · , ⁄m } the set of the eigenvalues (all distinct from each other) and, for every index i in J1, mK,
denote ri the multiplicity of ⁄i as a root of ‰u . The endomorphism u is diagonalizable if, and only if,
one the following conditions holds:

1. The characteristic polynomial ‰u splits in K (i.e. it factors completely in K) and for all i in
J1, mK, the equality dim(ker(u ≠ ⁄i IdE )) = ri holds.
m
ÿ
2. dim(ker(u ≠ ⁄i IdE )) = p. In other words, one can write:
i=1

m
ÿ
dim(E⁄i ) = dim(E) = p.
i=1

3. E = ker(u ≠ ⁄1 IdE ) + · · · + ker(u ≠ ⁄m IdE ). In other words, one can write:

E = E⁄1 + E⁄2 + · · · + E⁄m .

In this case, one can get a basis of E by gathering all the basis of every eigenspace ker(u ≠ ⁄i IdE ).

Proof 7.2.14
To be �lled! See [CPY96, p.140 Proposition V.14]
⇤

7.3 Diagonalizable Matrices

De�nition and Results

De�nition 7.3.1 (Diagonalizable Matrix)

A square matrix A of Mp (K) is said to be diagonalizable (on K) if the associated endomorphism
u : Kp æ Kp , de�ned by setting x ‘æ Ax, is diagonalizable.

We will keep, for matrices, the terminology we used for endomorphisms. We therefore call eigenvalue
(resp. eigenvector) of A every eigenvalue (resp. every eigenvector) of the associated endomorphism u.
The kernel of u ≠ ⁄IE , denoted ker(A ≠ ⁄Ip ), is called eigenspace of A, associated to the eigenvalue
⁄, and ‰A de�ned by setting ‰A (X) := det(A ≠ XIp ), is called the characteristic polynomial of A.

Theorem 7.3.2
A matrix A of Mp (K) is diagonalizable if, and only if there exists in Mp (K), an invertible matrix,
denoted P , and a diagonal matrix, denoted D, such that:

D = P ≠1 AP. (3.1)

94 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

Proof 7.3.3
To be �lled To be �lled! See [CPY96, p.141 Theorème V.16] ⇤

Remark 7.3.4
Matrices A and D are similar. Thus a matrix is diagonalizable if, and only if, it is similar to a diagonal
matrix.

For diagonalizability of square matrices, we have a result similar to 7.2.13.

Proposition 7.3.5 (Diagonalizability Criteria for Square Matrices)

Let A be a matrix of Mp (K). Denote {⁄1 , · · · , ⁄m } the set of the eigenvalues (all distinct from each
other) and, for every index i in J1, mK, denote ri the multiplicity of ⁄i as a root of ‰A . Matrix A is
diagonalizable if, and only if, one the following conditions holds:

1. The characteristic polynomial ‰A splits in K (i.e. it factors completely in K) and for all i in
J1, mK, the equality dim(ker(A ≠ ⁄i Ip )) = ri holds.
m
ÿ
2. dim(ker(A ≠ ⁄i Ip )) = p. In other words, one can write:
i=1

m
ÿ
dim(E⁄i ) = dim(Kp ) = p.
i=1

3. Kp = ker(A ≠ ⁄1 Ip ) + · · · + ker(A ≠ ⁄m Ip ). In other words, one can write:

Kp = E⁄1 + E⁄2 + · · · + E⁄m .

In this case, one can get a basis of Kp by gathering all the basis of every eigenspace ker(A ≠ ⁄i Ip ).

Proof 7.3.6
One just has to apply Proposition 7.2.13 with the endomorphism

u : Kp æ Kp
X ‘æ AX

Diagonalization in practice
Whenever A in Mp (K), is diagonalizable, diagonalize it means �nding an invertible matrix, denoted
P , and a diagonal matrix, denoted D such that the following equality holds:

D = P ≠1 AP

The matrix P is the change of basis matrix from the standard basis e (also called the former basis) to
a new basis, denoted f , which is constituted of the eigenvectors of A. Matrix D is the repsentation
matrix of the endomorphism u, from basis f to itself. In other words, D = [u]ff .
Starting from matrix A, we get matrices D and P by the following manner.

95 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

1. The eigenvalues of the matrix A are the roots of the characteristic polynomial ‰A (where ‰A (X) =
det(A ≠ XIp )).
2. We determine the eigenspaces of A and we choose a basis f := (f 1 , · · · , f p ) of Kp constituted
of eigenvectors of A (we obtain it by taking the union of the basis of the eigenspaces). The ith
column of the matrix P is then constituted of the components of f i in the standard basis of Kp .
3. We have D = diag(⁄1 , ·, ⁄p ) where the ⁄i are the eigenvectors of the matrix A. Each and every
one of them appear on the diagonal of D, a number of times which equals its multiplicity, as a
root of ‰A . They are ordered such that f i is associated to ⁄i .

Example 7.3.7 (Diagonalization)

Ç å
3 4
Denote A := the matrix we want to diagonalize.
5 2

1. We start by looking for the roots of its characteristic polynomial.

3≠X 4
‰u (X) = = (3 ≠ X)(2 ≠ X) ≠ 20 = X 2 ≠ 5X ≠ 14.
5 2≠X

It is easy to determine that the roots of ‰u are ≠2 and 7. Thus one can write:

‰u (X) = (X + 2)(X ≠ 7).

Since the characteristic polynomial of A has p = 2 distinct roots in K, we know, according to

Theorem 7.2.9 or Proposition 7.2.13, that A is diagonalizable.

2. We are now looking for the eigenvectors of A. By de�nition, we know that the two eigenvectors
are: E≠2 := Ker(A + 2I2 ) and E7 := Ker(A ≠ 7I2 ).

• Make E≠2 explicit

Let (x, y) be in R2 .
! ! !
5 4 x 0 5
(x, y) œ E≠2 ≈∆
t
· = ≈∆ 5x + 4y = 0 ≈∆ y = ≠ x
5 4 y 0 4

One then concludes that E≠2 = {(x, ≠ 54 x), x œ R} = SpanR {(≠4, 5)}. We can choose
x1 := (≠4, 5) as an eigenvector of A, associated to the eigenvalue ≠2. Note moreover
that (≠4, 5) is a basis of E≠2 .

• Make E7 explicit
Let (x, y) be in R2 .
! ! !
≠4 4 x 0
(x, y) œ E7 ≈∆
t
· = ≈∆ ≠4x + 4y = 0 ≈∆ y = x
5 ≠5 y 0

One then concludes that E7 = {(x, x), x œ R} = SpanR {(1, 1)}. We can choose
x2 := t(1, 1) as an eigenvector of A, associated to the eigenvalue 7. Note moreover that
(1, 1) is a basis of E7 .

96 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

Finally gathering the two basis of the eigenspaces, we know that B := (x1 , x2 ) is a basis of
R2 , constituted only with eigenvectors of A.
Besides, denote e := (e1 , e2 ) the standard basis R2 and let u : R2 æ R2 be the linear map,
the matrix representation (from e to e) of which is A. In other words, one has the equality:

A = [u]ee .

De�ne D := [u]B B . By de�nition of the xi , we know that u(x1 ) = ≠2x1 and that u(x2 ) = 7x2 .
It is therefore obvious that

u(x1 ) u(x2 )
Ç å
D = [u]B = ≠2 0 x1 .
B
0 7 x2

Moreover, denote P the change of basis matrix from e and B. By de�nition of P , one has the
equality:
x x2
Ç 1 å
P = [u]B =
e ≠4 1 e1 .
5 1 e2

Since A = [u]ee , we know, thanks to (3.14), that

D = [u]B
B = [u]e [u]e [u]B = P
B e e ≠1
AP. (3.2)

One can here verify that Equality holds (3.2) by �rst computing P ≠1 and then by computing
P ≠1 AP .
We easily get !
≠1/9 1/9
P ≠1 =
5/9 4/9
and then
! Ç å ! !
≠1/9 1/9 3 4 ≠4 1 ≠2 0
P ≠1 AP = · · =
5/9 4/9 5 2 5 1 0 7

which is nothing but D. We therefore recover the result which states that Equality (3.2) holds.

7.4 Applications
Computation of the nth power of a matrix

Let A be diagonalizable matrix. Equality (3.1) also reads

A = P DP ≠1 . (4.3)
It is therefore easy to deduce, that, for every n in N,

An = P DP ≠1 P DP ≠1 · · · P DP ≠1 = P Dn P ≠1 . (4.4)

97 S����� 2023
L����� A������ Lecture Notes C������ 7: Diagonalization

Matrix D (resp. A) represents the endomorphism u in the basis f , which is constituted with eigenvec-
tors of u (resp. in the standard basis e). Equality (4.4) expresses simply the fact that both matrices An
and Dn represent the same endomorphism (namely un ), respectively in the basis e and in the basis f .
Note moreover that the product of two diagonal matrices is very easy to compute. indeed, the product
matrix is nothing but a diagonal matrix, the ith term of which is the product of the ith terms of both
matrices. We therefore deduce that the diagonal matrix Dn is obtained by taking the nth power of each
and every of the factors on the diagonal of D.
Many other applications of diagonalization will be seen in Exercises. Note however that Diagonaliza-
tion can also be successfully used to solve di�erential linear system with constant coe�cients, to study
system of Constant-recursive sequences.
J’en suis à [CPY96, p.143 Applications]

98 S����� 2023
Part V

99
C������ 8

Isometries

100
C������ 9

Euclidean Spaces

101
Part VI

102
L����� A������ Lecture Notes C������ 8: Euclidean Spaces

1. PCA (ACP)

2. voir Strang et tout et tout!

3. Exponentielles de matrices

4. Résolution de systèmes di�éerentiels

5. Régression linéaires et machine learning

103 S����� 2023

Part VII

104
Appendix A

Mapping, Image and Preimage

A.1 Maps & Applications

De�nition A.1.1 (Map)

Let E and F be two non-empty sets. A map or a function f , from E to F is an assignment of exactly
one element of F to each element of E. Hence, one can de�ne a map by giving exactly four things:

1. the name of the map, here f

2. the departure set of the map, also called Domain, here E

3. the arrival set, also called co-domain, here F

4. the way to associate, to any element of E, a unique element of F .

If y0 is the unique element of F assigned by the function f to the element x0 of E, it is written as

f (x0 ) = y0 . f maps E to F means that f is a function from E to F , it is written as f : E æ F .

Example A.1.2
Let E := {a, b, c, d}, F := {1, 2, 3, 4} and let F Õ be the set de�ned by setting F Õ := {1, 2, 3, 4, 5}.
De�ne the maps f, g and h by setting:

h : E æ F
f : E æ F g : E æ F
a ‘ æ 4
a ‘ æ 4 a ‘ æ 2
a ‘ æ 1
b ‘ æ 2 , b ‘ æ 2 & .
b ‘ æ 2
c ‘ æ 1 c ‘ æ 1
c ‘ æ 1
d ‘ æ 3 d ‘ æ 3
d ‘ æ 3

One can represent these maps in the following manner:

105
L����� A������ Lecture Notes C���.A M��� & A�����������

E F E F E F
f g h
a 1 a 1 a 1
b 2 b 2 b 2
c 3 c 3 c 3
d 4 d 4 d 4

Figure A.1: Mapping diagrams of f , g and h.

Both maps f and g are correctly de�ned. However, h is not correctly de�ned since h(a) = 1 and
h(a) = 4.
Let us also consider the maps s, i and k, from E to F Õ , de�ned by their graph, given below:

E FÕ E FÕ E FÕ
s i k
a 1 a 1 a 1
b 2 b 2 b 2
5 5 5
c 3 c 3 c 3
d 4 d 4 d 4

Figure A.2: Mapping diagrams of s, i and k.

Both maps s and i are correctly de�ned. However, k is not correctly de�ned since k(a) = 1 and
k(a) = 4.

Example A.1.3
Let g, h, Ï and  be the maps de�ned by setting:

g : R æ R h : R æ R≠ ú Ï : R+ æ R+ Â : R æ R+
x ‘æ x2 x ‘æ x2 x ‘æ x2 x ‘æ x2

It is clear that the maps g, Ï and  are correctly de�ned while h is not a map since it is impossible
for the square of a real number to be negative.

Terms related to functions

Let
f :EæF (1.1)
be a map. Let us see the terms related to the de�nition of a function.
Domain and co-domain:
If f is a function from set E to set F , then E is called Domain and F is called co-domain.
Range:
The range of f is the set of all images of elements of E. Basically the range is subset of the co-domain.
One can de�ne it by
Range(f ) := {f (u), u œ E}.

106 S����� 2023

L����� A������ Lecture Notes C���.A M��� & A�����������

Image and Pre-Image:

For any (a, b) in E ◊ F (which means that a belongs to E and b belongs to F ), b is the image of a and
a is the pre-image of b if f (a) = b.
One can extend these last de�nitions to the cases of sets. More precisely, if we still consider the map f
introduced in (1.1), one can de�ne:

- for every subset A of E the set

f (A) := {f (u), u œ A}, (1.2)

which is called the image of A by f . Note that f (A) is a subset of F .

- for every subset B of F the set

f ≠1 (B) := {u œ E, f (u) œ B}, (1.3)

which is called the preimage of B by f . Note that f ≠1 (B) is a subset of E.

Example A.1.4
Let us also consider the map · : E æ F , the graphic representation of which is given below:

E F
· B
a 1
b 2
c 3
d 4

Figure A.3: Mapping diagram of · .

De�ne A := {a, c} and B := {1, 2}. We have the following equalities:

· (A) = {· (x), x œ A} = {· (a), · (c)} = {1, 4},

&
· ≠1 (B) = {x œ E, s.t. · (x) œ B} = {x œ E, s.t. · (x) = 1 or · (x) = 2} = {b, c}.

Remark A.1.5
From these previous de�nitions, one easily sees that the range of function is noting but the image of
its domain. In other words, for any map f : E æ F , one has: range(f ) = f (E).

A.2 Injectivity, Surjectivity and Bijectivity

There are three particular types of functions that are of great interest.

107 S����� 2023

L����� A������ Lecture Notes C���.A M��� & A�����������

De�nition A.2.1 (One to one or injective function)

Let E and F be two non-empty sets. A amp or function f : E æ F is called one to one (or injective)
if, for all elements x and xÕ in E, if f (x) = f (xÕ ), then x = xÕ . It never maps distinct elements of its
domain to the same element of its co-domain. If we use quanti�ers, f is injective if,

’(x, xÕ ) œ E 2 , f (x) = f (xÕ ) =∆ x = xÕ . (2.4)

Remark A.2.2
An alternative characterization of injectivity, which is an alternative to (2.4) is:

’y œ F, there exists at most one x in E such that y = f (x). (2.5)

Example A.2.3
In Example A.1.2 above, only f and i are injective functions.

Example A.2.4
If one considers the maps g, h, Ï and  de�ned in Example A.1.3, it is easy to see that:

• g is not injective since g(≠3) = 9 = g(3).

• Ï is injective since, for all (a, b) in R+ 2 , one has:

Ï(a) = Ï(b) =∆ a2 = b2 =∆ (a ≠ b) (a + b) = 0 =∆ a = b or a = ≠b.

Since both a and b are positive, the case a = ≠b is impossible and thus we conclude that

’(a, b) œ R+ 2 , Ï(a) = Ï(b) =∆ a = b.

This proves the injectivity of Ï.

• Â is not injective since Â(≠3) = 9 = Â(3).

De�nition A.2.5 (Onto or surjective function)

Let E and F be two non-empty sets. A function f : E æ F is called onto or surjective if all element
of the codomain has a preimage in the domain. In other words f is surjective if, for all y of F there
exists an element x in E such that y = f (x).
The function f may map one or more elements of E to the same element of F . If we use quanti�ers,
f is surjective if,

’y œ F, ÷x œ E, s.t. y = f (x). (2.6)

Example A.2.6
In Example A.1.2 above, only f is a surjective functions.

108 S����� 2023

L����� A������ Lecture Notes C���.A M��� & A�����������

Example A.2.7
If one still considers the maps g, h, Ï and  de�ned in Example A.1.3, it is easy to see that:

• g is not surjective since, e.g. ≠4, does not have a preimage. Indeed, ≠4 = x2 has no real
solution. For the exact same reason h is not surjective either.
Ô
• Both
Ô Ï and  are surjective since all b in R+ has a preimage in R+ . Indeed, Â( b) = b =
Ï( b), for all b in R+ .

We have the following result

Theorem A.2.8
Let f : E æ F be a map. The two following statements are equivalent:

1. f is surjective

2. f (E) = F

Proof A.2.9
To be �lled! ⇤

De�nition A.2.10 (One to one correspondence or Bijective function)

Let E and F be two non-empty sets. A function f : E æ F is called bijective if it is both one to one
and onto function (i.e. both injective and surjective function).
For such a function, there exists a map, denoted f ≠1 : F æ E, characterized by the folloing equali-
ties:
’x œ E, f ≠1 (f (x)) = x, (2.7)

’y œ F, f (f ≠1 (y)) = y. (2.8)
We usually call f ≠1 the inverse function of f .

Example A.2.11
In Example A.1.2 above, only f is a bijective function.

Example A.2.12
If one still considers the maps g, h, Ï and  de�ned in Example A.1.3, it is clear, in view of Examples
A.2.4 and A.2.7, that only Ï is bijective. Its inverse is the map Ï≠1 : R+ æ R+ de�ned by: Ï≠1 (u) :=
Ô
u, for every u in R+ .

Remark A.2.13
We have to be extremely cautious with the notation f ≠1 . Indeed, this notation can be used in two
very di�erent situations. Either one wants to speak about:

• the subset f ≠1 (B) of E, where B is a subset of F (as de�ned in (1.3)). Such a set always exists
(i.e. we do not need f to be bijective).

• the inverse function of f , which is possible only if f is bijective.

109 S����� 2023

L����� A������ Lecture Notes C���.A M��� & A�����������

Hence, in example A.1.4, we made explicit the set · ≠1 (B) while · is not bijective and therefore · ≠1
(i.e. the inverse of · ) does not exist.

Example A.2.14
Let f : E æ F be an application. Show that:
(i) f is injective if and only if ’A µ E, A = f ≠1 (f (A)).
(ii) f is surjective if and only if ’C µ F, f (f ≠1 ((C)) = C.
(iii) f is bijective if and only if:
a) ’A µ E, A = f ≠1 (f (A)),
b) ’C µ F, f (f ≠1 ((C)) = C.
Let A be a subset of E and C be a subset of F . By de�nition we have:

f (A) := {f (x), x œ A} & f ≠1 (C) := {x œ E, f (x) œ C}. (E1 )

It is therefore clear that:

A µ f ≠1 (f (A)) and f (f ≠1 (C)) µ C, (E2 )

for all subsets A of E and C of F .
(i) We have to show that f is injective if and only A = f ≠1 (f (A)), for all subset A of E.
• Assume that f is injective and let A be a subset E. In view of (E2 ), we only have to show that
f ≠1 (f (A)) µ A. Let x be an element of f ≠1 (f (A)). By de�nition, f (x) belongs to f (A). By
de�nition of f (A), this implies in particular that there exists z in A such that
f (x) = f (z).
Since f injective this implies that x = z. Since z belongs to A, so does x.
• Assume that A = f ≠1 (f (A)), for all subset A of E and let x and y be two elements of E such that
f (x) = f (y). Our goal is to show that x = y.
De�ne A := {y}. Note that f ({y}) = {f (y)}. Moreover, since f (x) = f (y), it is clear that x belongs
to f ≠1 ({f (y)}) = f ≠1 (f ({y})) = {y}. Thus x = y.
(ii) We have to show that f is surjective if and only f (f ≠1 (C)) = C, for all subset C of F .
• Assume that f is surjective and let C be a subset F . In view of (E2 ), we only have to show
that C µ f (f ≠1 (C)). Let y be an element of C. Since f is surjective, there exists x in E such
that y = f (x). Since f (x) belongs to C, y belongs to {f (u), u œ f ≠1 (C)}, which is nothing but
f (f ≠1 (C)).
• Assume that f (f ≠1 (C)) = C, for all subset C of F . Our goal is to show that f is surjective.
Let y be an element of F . We are looking for an x in E such that y = f (x). By assumption, we know
that f (f ≠1 ({y})) = {y}. Since y belongs to f (f ≠1 ({y})), we know that there exists u in f ≠1 ({y}
(and thus in E) such that y = f (u). De�ne x := u. We get y = f (x) where x belongs to E.
(iii) We have to show that f is bijective if and only:
a) ’A µ E, A = f ≠1 (f (A)),
b) ’C µ F, f (f ≠1 ((C)) = C.
This is obvious, in view of (i) and (ii), since bijectivity is both injectivity and surjectivity. ⇤.

110 S����� 2023

Bibliography

[CPY96] Gilles Christol, Philippe Pilibossian, and Sleiman Yammine. Algèbre 2 - Espaces vectoriels,
matrices, systèmes linéaires. Ellipses, 1996.

[Gou09] Xavier Gourdon. Les maths en tête. Algèbre - 2e édition. Les maths en tête. Ellipses, 2009.

[HJ13] Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, Cam-
bridge, second edition, 2013.

[Str16] Gilbert Strang. Introduction to Linear Algebra. Wellesley-Cambridge Press; 5th edition, 2016.

111
Index

Matrices
Similar matrices, 81
Matrix
Change of basis -, 75
Equivalent matrices, 79
nilpotent, 7

Rank
-Nullity Theorem, 68
of a family of vectors, 67
of a linear map, 67
of a matrix, 17

Theorem
Rank-Nullity, 68

112
List of Symbols

Mp,q (K): the set of all p ◊ q matrices with coe�cients in K, 2

GLp (K): General linear group of degree p, 9

K: denotes either the set of real numbers R or the set of complex numbers C, 2

J1, pK: intervall of all integers from 1 to p, 1 and p being both included, 2

113