Inner product space

Dr Abhay Kumar Singh

December 4, 2024

Dr Abhay Kumar Singh Inner product space December 4, 2024 1 / 19

Outline

1 Introduction

2 Orthogonality

3 Gram-Schmidt orthogonalization

4 Orthogonal Complement

5 Orthogonal matrices

Dr Abhay Kumar Singh Inner product space December 4, 2024 2 / 19

Introduction

Inner product
Let F be the field of real numbers or the field of complex numbers, and V
be a vector space over F . An inner product on V is a function that assigns
to each ordered pair of vectors α, β ∈ V a scalar (α|β) ∈ F in such a way
that the following properties hold for all α, β, γ ∈ V and all scalars c:
(α + β|γ) = (α|γ) + (β|γ) (Linearity in the first argument)
(cα|β) = c(α|β) (Homogeneity in the first argument)
(β|α) = (α|β) (Conjugate symmetry)
(α|α) > 0 if α ̸= 0 (Positive definiteness)

Dr Abhay Kumar Singh Inner product space December 4, 2024 3 / 19

Example

Example:Let V = R2 , the vector space of all ordered pairs of real

numbers, and define the inner product for any u = (u1 , u2 ) and
v = (v1 , v2 ) in V as:
⟨u, v⟩ = u1 v1 + u2 v2 .
Verification of Inner Product Properties
1. Linearity in the First Argument: For any u, v, w ∈ V and scalar c, we
have:

⟨u+v, w⟩ = (u1 +v1 )w1 +(u2 +v2 )w2 = u1 w1 +v1 w1 +u2 w2 +v2 w2 = ⟨u, w⟩+⟨v

Similarly,

⟨cu, v⟩ = (cu1 )v1 + (cu2 )v2 = c(u1 v1 + u2 v2 ) = c⟨u, v⟩.

Dr Abhay Kumar Singh Inner product space December 4, 2024 4 / 19

2. Symmetry:

⟨u, v⟩ = u1 v1 + u2 v2 = v1 u1 + v2 u2 = ⟨v, u⟩.

3. Positive Definiteness:

⟨u, u⟩ = u12 + u22 ≥ 0,

with equality if and only if u = (0, 0).

Thus, V = R2 with the defined inner product is an inner product space.

Dr Abhay Kumar Singh Inner product space December 4, 2024 5 / 19

Orthogonality

Orthogonal Vectors
Let α and β be vectors in an inner product space V . Then α is orthogonal
to β if (α|β) = 0

Orthogonal Set
If S is a set of vectors in V , S is called an Orthogonal Set provided all
pairs of distinct vectors in S are orthogonal.

Orthonormal Set
An Orthonormal Set is an orthogonal set S with an additional property
that ||α|| = 1 for every α in S.

Note: The zero vector is orthogonal to every vector in V.

Dr Abhay Kumar Singh Inner product space December 4, 2024 6 / 19

Orthogonality Example

Here are examples of orthogonal sets and orthogonal vectors

Orthogonal Vectors:
Two vectors u and v in Rn are orthogonal if their dot product is zero, i.e.,
u · v = 0.
Let,    
1 2
u = −2 , v = 1
  
1 2
To verify:

u · v = 1 · 2 + (−2) · 1 + 1 · 2 = 2 − 2 + 2 = 0

Thus, u and v are orthogonal.

Dr Abhay Kumar Singh Inner product space December 4, 2024 7 / 19

Orthogonal Set:
A set of vectors {v1 , v2 , . . . , vn } is orthogonal if every pair of distinct
vectors in the set is orthogonal, i.e., vi · vj = 0 for i ̸= j. Let,
     
1 0 0
v1 = 0 , v2 = 1 , v3 = 0
0 0 1

To verify:
v1 · v2 = 1 · 0 + 0 · 1 + 0 · 0 = 0
v1 · v3 = 1 · 0 + 0 · 0 + 0 · 1 = 0
v2 · v3 = 0 · 0 + 1 · 0 + 0 · 1 = 0
Thus, {v1 , v2 , v3 } is an orthogonal set.

Dr Abhay Kumar Singh Inner product space December 4, 2024 8 / 19

Theorem

Theorem 1
An orthogonal set of non-zero vectors is linearly independent.

Proof. Let {v1 , v2 , . . . , vn } be an orthogonal set of non-zero vectors in a

vector space. Assume a linear combination of these vectors equals the zero
vector:
c1 v1 + c2 v2 + · · · + cn vn = 0,
where c1 , c2 , . . . , cn ∈ R.
For any i ∈ {1, 2, . . . , n}, we take the dot product of both sides of the
equation with vi :

(c1 v1 + c2 v2 + · · · + cn vn ) · vi = 0 · vi .

Dr Abhay Kumar Singh Inner product space December 4, 2024 9 / 19

Simplifying the left-hand side using the linearity of the dot product:

c1 (v1 · vi ) + c2 (v2 · vi ) + · · · + cn (vn · vi ) = 0.

Since the set is orthogonal, vj · vi = 0 for j ̸= i. Moreover,

vi · vi = ∥vi ∥2 > 0 because vi ̸= 0.
Thus, the equation reduces to:

ci ∥vi ∥2 = 0.

Since ∥vi ∥2 > 0, it follows that:

ci = 0.

By applying the same argument for each i ∈ {1, 2, . . . , n}, we conclude

that:
c1 = c2 = · · · = cn = 0.
Since the only solution to the equation is the trivial solution, the set
{v1 , v2 , . . . , vn } is linearly independent.
Dr Abhay Kumar Singh Inner product space December 4, 2024 10 / 19
Gram-Schmidt orthogonalization process

Theorem 2
Let V be an inner product space and v1 , . . . , vn be any linearly
independent vectors in V. Then one may construct orthogonal vectors
u1 , . . . , un in V such that the set {u1 , . . . , un } is a basis for the subspace
spanned by v1 , . . . , vn .

The vectors u1 , . . . , un will be obtained by means of a construction known

as the Gram-Schmidt orthogonalization process.
Gram-Schmidt Orthogonalization Process:
Given a linearly independent set {v1 , v2 , . . . , vn }.
1. We start with the first vector:

u1 = v1 .

Dr Abhay Kumar Singh Inner product space December 4, 2024 11 / 19

2. For k = 2, 3, . . . , n, we compute:
k−1
X vk · uj
uk = vk − uj .
uj · uj
j=1

3. Then, we normalize each uk to get an orthonormal vector ek :

uk
ek = .
∥uk ∥

Corollary
Every finite-dimensional inner product space has an orthonormal basis.

Dr Abhay Kumar Singh Inner product space December 4, 2024 12 / 19

Orthogonal Complement

Orhogonal Complement
Let S be a nonempty subset of an inner product space V . We denote by
S ⊥ , the set of all vectors of V that are orthogonal to every vector of S,
called the orthogonal complement of S in V . In notation,

S ⊥ = {v ∈ V | ⟨v, u⟩ = 0 for all u ∈ S}.

If S contains only one vector u, we write

u⊥ = {v ∈ V | ⟨v, u⟩ = 0}.

Dr Abhay Kumar Singh Inner product space December 4, 2024 13 / 19

Orthogonal Subspace

Proposition
Let S be a nonempty subset of an inner product space V . Then the
orthogonal complement S ⊥ is a subspace of V .

1. Zero Vector Belongs to S ⊥ : We know that for the zero vector 0,

⟨0, v⟩ = 0 for all v ∈ S. Hence, 0 ∈ S ⊥ .

2. Closure under Addition: Let x, y ∈ S ⊥ . Then, for any v ∈ S:

⟨x, v⟩ = 0 and ⟨y, v⟩ = 0.

Dr Abhay Kumar Singh Inner product space December 4, 2024 14 / 19

We need to show that x + y ∈ S ⊥ . Consider:

⟨x + y, v⟩ = ⟨x, v⟩ + ⟨y, v⟩.

Since ⟨x, v⟩ = 0 and ⟨y, v⟩ = 0, it follows that:

⟨x + y, v⟩ = 0.

Therefore, x + y ∈ S ⊥ .
3. Closure under Scalar Multiplication: Let x ∈ S ⊥ and a ∈ R (or C,
depending on the field of V ). We need to show that ax ∈ S ⊥ . For any
v ∈ S:
⟨ax, v⟩ = a⟨x, v⟩.
Since ⟨x, v⟩ = 0, it follows that:

⟨ax, v⟩ = a · 0 = 0.

Thus, ax ∈ S ⊥ .

Dr Abhay Kumar Singh Inner product space December 4, 2024 15 / 19

Example

Let A be an m × n real matrix. Then Nul A and Row A are orthogonal

complements of each other in Rn , i.e.,

Nul A = (Row A)⊥ ,

(Nul A)⊥ = Row A.

Dr Abhay Kumar Singh Inner product space December 4, 2024 16 / 19

Orthogonal matrices

Definition
A square matrix A is said to be an orthogonal matrix if it has orthonormal
columns
Note:For square orthogonal matrices, the transpose is the inverse i.e.

A−1 = AT

=⇒ AAT = AT A = I
When row i of AT multiplies column j of A, the result is qiT qj = 0. On the
diagonal where i = j, we have qiT qi = 1. That is the normalization to unit
vectors of length 1.

Dr Abhay Kumar Singh Inner product space December 4, 2024 17 / 19

Orthogonal matrices examples
These are some examples of Orthogonal Matrices:
 
cosθ sinθ
1. Rotation matrices are orthogonal matrices, A =
−sinθ cosθ
In this case the transpose is:
 
T cosθ −sinθ
A =
sinθ cosθ
 
0 1 0
2. Permutation matrices are orthogonal matrices, for e.g A = 1 0 0
0 0 1
In this case, the transpose is:
 
0 1 0
AT = 1 0 0
0 0 1

Dr Abhay Kumar Singh Inner product space December 4, 2024 18 / 19

 THANK YOU

Dr Abhay Kumar Singh Inner product space December 4, 2024 19 / 19