CHAPTER 5.

INNER PRODUCT SPACES

AND LINEAR OPERATORS
MODULE 2. ORTHOGONALIZATION AND ORTHOGONAL
COMPLEMENT

INDRANATH SENGUPTA

Contents
1. Orthogonal and Orthonormal sets of vectors 1
2. Gram-Schmidt process of orthogonalization 3
3. Orthogonal complement 4

We assume that the field F is either R or C. Let V be an inner

product space over F.

1. Orthogonal and Orthonormal sets of vectors

Definition 1. Two vectors x and y in an inner product space V are
said to be orthogonal if hx, yi = 0. A non-empty subset S of V is said
to be orthogonal if hx, yi = 0 for all x 6= y in S.
A non-empty subset S of V is said to be orthonormal if S is orthogonal
and ||x|| = 1 for all x in S.

Example 1.1. Let V = Fn . The standard basis {e1 , . . . , en } is an

orthonormal set in V .

Example 1.2. If {v1 , . . . , vm } is an orthogonal set in V then the set

of vectors {c1 v1 , . . . , cm vm } is also an orthogonal set in V , for scalars
c1 , . . . , cm in F.

Example 1.3. Let V = R3 . The set {(2, 1, 0), (−2, 1, 0), (0, 0, 1)} is
an orthogonal set in V , which is not orthonormal. We can make it
orthonormal by scaling each vector by their norm. Therefore, the set
{ √15 (2, 1, 0), √15 (−1, 2, 0), (0, 0, 1)} is an orthonormal set in V .
1
2 Module 2
Example 1.4. The vector space of all complex-valued continuous func-
tions on [0, 2π], denoted by V = C[0, 2π]C has an orthonormal set

S = {n (t) = cos nt + i sin nt | n ∈ Z}.

Definition 2. Let V be an inner product space over F. A subset B of

V is called an orthonormal basis (onb in short) if B is an ordered basis
for V , which is orthonormal.

Example 1.5. The standard basis {e1 , . . . , en } of Fn and the subset

{ √15 (2, 1, 0), √15 (−1, 2, 0), (0, 0, 1)} of R3 are examples of orthonormal
bases.

Example 1.6. The subset S = {n (t) = cos nt + i sin nt | n ∈ Z} of

the inner product space V = C[0, 2π]C is orthonormal. This set can
not be a basis for the vector space V . However, given f ∈ V , the scalar
cn = hf, n i is called the n-th Fourier coefficient of the continuous
function f relative to the orthonormal set S.

Proposition 1.7. (i) Every orthogonal set of non-zero vectors in

an inner product space is linearly independent.

(ii) Let S = {v1 , . . . , vm } be a an orthonormal subset of the inner

P space V over F. Let x ∈ V belong to span(S). Then,
product
x= m i=1 hx, vi ivi .

Proof. (i) Let c1 v1 + · · · + cm vm = 0, for c1 , . . . , cm ∈ F. Then, for

every i,
hc1 v1 + · · · + cm vm , vi i = ci hvi , vi i = 0,
since hvi , vj i = 0 if i 6= j. Hence, ci = 0, since ||vi ||2 = hvi , vi i 6= 0 for
every i.

(ii) Let x = c1 v1 + · · · + cm vm = 0, for c1 , . . . , cm ∈ F. Then hx, vi i = ci

for all i, since hvi , vj i = 0 if i 6= j and ||vi ||2 = hvi , vi i = 1. 
Chapter 5. Inner Product Spaces and Linear Operators 3

2. Gram-Schmidt process of orthogonalization

Gram-Schmidt Process of orthogonalization. Let {x1 , x2 } be a
linearly independent subset of an inner product space V . Let W denote
the span of the set {x1 , x2 }. Consider the vector v = x2 − cx1 ∈ W .
We want to choose a scalar c in F, such that hv, x1 i = 0, that is,
0 = hv, x1 i = hx2 − cx1 , x1 i = hx2 , x1 i − chx1 , x1 i.
hx2 ,x1 i hx2 ,x1 i
Therefore, c = hx1 ,x1 i
= ||x1 ||2
is the scalar and the set {x1 , x2 −
hx2 ,x1 i
x}
is an orthogonal basis for the subspace W . If we scale the
||x1 ||2 1
vectors by their norms then we obtain an orthonormal basis for W .
This is called the process of orthogonalization. The theorem below
gives the generalized version.
Theorem 2.1 (Gram-Schmidt). Let V be an inner product space
over F. Let S = {x1 , . . . , xn } be a linearly independent subset of V .
Define v1 = x1 and
k−1
X hxk , vj i
v k = xk − vj for 2 ≤ k ≤ n.
j=1
||vj ||2
Then, S 0 = {v1 , . . . , vn } is an orthogonal set of vectors and span(S) =
span(S 0 ).

Proof. The proof is by induction on n; see Theorem 6.3 in Friedberg

et.al. 
Corollary 2.2. Every non-zero finite dimensional inner product space
has an orthonormal basis.

Example 2.3. Let V = R2 [x] the vector space of all polynomials over
R with degree at most 2. We define
Z 1
hp(x), q(x)i = p(t)q(t)dt.
−1

Then, V is an inner product space. Consider the standard basis {1, x, x2 }

of V . We apply the Gram-Schmidt process to create an orthogonal ba-
sis for V and it is given by the set {1, x, x2 − 13 }.
We can do the same process starting with the standard basis {1, x, x2 , . . .}
of the inner product space V = R[x] and obtain an orthogonal ba-
sis for V . The new polynomials obtained by this method are called
the Legendre polynomials. The polynomials p1 (x) = 1, p2 (x) = x,
p3 (x) = x2 − 13 written above are the first three of them.
4 Module 2

3. Orthogonal complement
Definition 3. Let V be an inner product space over F. Let S be a
non-empty subset of V . The orthogonal complement of the set S is
defined as
S ⊥ = {x ∈ V | hx, vi = 0 for all v ∈ V };
usually read as “S perp”.

Example 3.1. Let V = Rn . Then {en }⊥ is the subspace spanned by

{e1 , . . . , en−1 }. In particular, if n = 2, then {e1 }⊥ is the y-axis and
{e2 }⊥ is the x-axis.

Proposition 3.2. (i) {0}⊥ = V ;

(ii) V ⊥ = {0};
(iii) S ⊥ is a subspace of V .

Proof. Follows from the definition of S ⊥ . 

Theorem 3.3. Let V be an inner product space over F. Let W be a fi-

nite dimensional subspace of V with an orthonormal basis {v1 , . . . , vm }.
⊥
Let v ∈ V . There exit unique Pm vectors x ∈ W and y ∈ W , such that
v = x + y. Moreover, x = i=1 hx, vi ivi .
Pm ⊥
Proof.
Pm The vector y = v − i=1 hx, vi ivi belongs to W , because hv −
i=1 hx, vi ivi , vj i = 0 for every 1 ≤ j ≤ n. The uniqueness of x and y
follow from the fact that W ∩ W ⊥ = {0}. 

Remark 3.4. The unique vector x ∈ W is called the orthogonal projec-

tion of v on W . Moreover, the vector x is the closest to v, in the sense
that ||v − x|| ≤ ||v − w|| for every w ∈ W . For,
||v − w||2 = ||x + y − w||2
= ||(x − w) + y||2
= ||x − w||2 + ||y||2
≥ ||y||2
= ||v − x||2 .

Theorem 3.5. Let V be an inner product space of dimension n over F.

Let B = {v1 , . . . , vm } be an orthonormal set in V , with W = span(B).
Then
Chapter 5. Inner Product Spaces and Linear Operators 5

(i) B can be extended to an orthonormal basis B of V such that

B = B ∪ B 0 and B 0 is an orthonormal basis of W ⊥ .
(ii) dim(V ) = dim(W ) + dim(W ⊥ ).
Proof. (i) We know that by replacement theorem B can be extended
to an ordered basis {v1 , . . . , vm , vm+1 , . . . , vn } for V . By the Gram-
Schmidt process we can orthogonalize {v1 , . . . , vm , vm+1 , . . . , vn } to pro-
duce the basis B for V .
In this process, the first m elements of B are nothing but v1 , . . . , vm ,
since they were orthonormal to begin with. The basis B is therefore an
extension of the basis B of W . Moreover, {vm+1 , . . . , vn } is a linearly
independent subset of W ⊥ . Let v ∈ W ⊥ . We can write
Xn
v= hv, vi ivi .
i=1
Pn
We have hv, vi i = 0 for all i = 1, . . . , m. Hence, v = i=m+1 hv, vi ivi .
Therefore, {vm+1 , . . . , vn } is a basis for W ⊥ .
(ii) It follows that dim(V ) = n = m + (n − m) = dim(W ) + dim(W ⊥ ).