MA 106: Linear Algebra: Prof. B.V. Limaye IIT Dharwad
MA 106: Linear Algebra: Prof. B.V. Limaye IIT Dharwad
Lecture 21
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Inner Product Spaces
In Lecture 12, we introduced the inner product
⟨x, y⟩ := x∗ y = x 1 y1 + · · · + x n yn
[ ]T
of a column vector x := x1 · · · xn ∈ Kn×1 with a column
[ ]T
vector y := y1 · · · yn ∈ Kn×1 . We also pointed out (and
used) some crucial properties of the inner product function
⟨· , ·⟩ : Kn×1 × Kn×1 → K.
The above definition of an inner product does not make sense
for elements of an abstract vector space. In fact, not all vector
spaces have usable inner product functions.
We, therefore, focus on those vector spaces for which there is
a function having the crucial properties of the inner product
on Kn×1 mentioned above. . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Let V be a vector space over K. An inner product on V is
function ⟨· , ·⟩ : V ×V → K satisfying the following properties.
For u, v , w ∈ V and α ∈ K,
1. ⟨v , v ⟩ ≥ 0 and ⟨v , v ⟩ = 0 ⇐⇒ v = 0, (positive definite)
2. ⟨u, αv + βw ⟩ = α⟨u, v ⟩+β⟨u, w ⟩, (linear in 2nd variable)
3. ⟨v , u⟩ = ⟨u, v ⟩. (conjugate symmetric)
From the above properties, conjugate linearity in the 1st
variable follows: ⟨αu + βv , w ⟩ = α⟨u, w ⟩ + β⟨v , w ⟩.
If u, v ∈ V and ⟨u, v ⟩ = 0, then we say that u and v are
orthogonal, and we write u ⊥ v .
For v ∈ V , we define the norm of v by ∥v ∥ := ⟨v , v ⟩1/2 .
Clearly, v ⊥ v ⇐⇒ ∥v ∥ = 0 ⇐⇒ v = 0.
If v ∈ V and ∥v ∥ = 1, then we say that v is a unit vector or
a unit function. The set {v ∈ V : ∥v ∥ ≤ 1} is called the
unit ball of V . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A vector space V over K with a prescribed inner product on it
is called an inner product space.
Examples
1 . We have already studied the primary example, namely
V := Kn×1 with the usual inner product ⟨x, y⟩ := x∗ y for
x, y ∈ Kn×1 . There are other inner products on Kn×1 . For
example, let w1 , . . . , wn be positive real numbers, and define
⟨w , v ⟩
⟨w , v − Pw (v )⟩ = ⟨w , v ⟩−⟨w , Pw (v )⟩ = ⟨w , v ⟩− ⟨w , w ⟩ = 0.
⟨w , w ⟩
∥v ∥2 = ⟨v , v ⟩ = ⟨Pw (v ), Pw (v ) + u⟩ + ⟨u, Pw (v ) + u⟩
= ∥Pw (v )∥2 + ∥u∥2 (by (i) above)
≥ ∥Pw (v )∥2 .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Theorem
Let ⟨· , ·⟩ be an inner product on a vector space V , and let
v , w ∈ V . Then
(i) (Schwarz Inequality) |⟨v , w ⟩| ≤ ∥v ∥∥w ∥.
(ii) (Triangle Inequality) ∥v + w ∥ ≤ ∥v ∥ + ∥w ∥.
Proof.
(i) Let w = 0. Then ⟨v , w ⟩ = ⟨v , 0⟩ = ⟨v , 0 + 0⟩ = 2⟨v , 0⟩,
and so ⟨v , w ⟩ = 0. Also, ∥w ∥ = 0. Hence we are done.
Now suppose w ̸= 0. Then by (ii) of the previous proposition,
⟨w , v ⟩
⟨w , w ⟩ w
= ∥Pw (v )∥ ≤ ∥v ∥,
∥v + w ∥2 = ⟨v + w , v + w ⟩ = ∥v ∥2 + ∥w ∥2 + 2 R ⟨v , w ⟩
≤ ∥v ∥2 + ∥w ∥2 + 2 |⟨v , w ⟩|
≤ ∥v ∥2 + ∥w ∥2 + 2 ∥v ∥∥w ∥ (by (i) above)
= (∥v ∥ + ∥w ∥)2 .
Thus ∥v + w ∥ ≤ ∥v ∥ + ∥w ∥.
We observe that the norm function ∥ · ∥ : V → K satisfies the
following three crucial properties:
(i) ∥v ∥ ≥ 0 for all v ∈ V and ∥v ∥ = 0 ⇐⇒ v = 0,
(ii) ∥αv ∥ = |α|∥v ∥ for all α ∈ K and v ∈ V ,
(iii) ∥v + w ∥ ≤ ∥v ∥ + ∥w ∥ for all v , w ∈ V .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Let V be an inner product space.
The Pythagorus Theorem says that if v , w ∈ V and v ⊥ w ,
then ∥v + w ∥2 = ∥v ∥2 + ∥w ∥2 . The proof is exactly as before.
Let E be a subset of V . Define
E ⊥ := {w ∈ V : w ⊥ v for all v ∈ E }.
Example
Let V be the set of all real-valued polynomial functions on
[−1, 1] along with the inner product defined by
∫ 1
⟨p, q⟩ := p(t)q(t)dt for p, q ∈ V .
−1
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Orthogonal Projections
Let W be a subspace of a finite dimensional inner product
space V . The Orthogonal Projection Theorem says that
for every v ∈ V , there are unique w ∈ W and w̃ ∈ W ⊥ such
that v = w + w̃ , that is, V = W ⊕ W ⊥ . The map
PW : V → V given by PW (v ) = w is linear and satisfies
(PW )2 = PW .
In fact, if u1 , . . . , uk is an orthonormal basis for W , then
PW (v ) = ⟨u1 , v ⟩u1 + · · · + ⟨uk , v ⟩uk and w̃ = v − w .
The linear map PW : V → V is called the orthogonal
projection of V onto the subspace W .
Given v ∈ V , its orthogonal projection PW (v ) into W is the
unique best approximation to v from W .
Further, PW (v ) is the unique element of W such that
v − PW (v ) is orthogonal to W . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
.
.
.
.
. .
. .
.
T (v1 ) = ⟨u1 , T (v1 )⟩u1 +· · · +⟨uj , T (v1 )⟩uj + · · · +⟨um , T (v1 )⟩um ,
.. .. ..
. . .
T (vk ) = ⟨u1 , T (vk )⟩u1 +· · · +⟨uj , T (vk )⟩uj + · · · +⟨um , T (vk )⟩um ,
.. .. ..
. . .
T (vn ) = ⟨u1 , T (vn )⟩u1 +· · · +⟨uj , T (vn )⟩uj + · · · +⟨um , T (vn )⟩um .
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Hence the m × n matrix of the linear map T with respect to
the basis E of V and the basis F of W is
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Using the above result, it is possible to prove the spectral
theorem for a Hermitian operator on a finite dimensional inner
product space V . Moreover, one can define the adjoint T ∗ of
a linear operator T on V , and prove a spectral theorem for an
operator T which commutes with its adjoint T ∗, that is, for a
normal operator on V . However, we shall stop here! Finally!
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
Using the above result, it is possible to prove the spectral
theorem for a Hermitian operator on a finite dimensional inner
product space V . Moreover, one can define the adjoint T ∗ of
a linear operator T on V , and prove a spectral theorem for an
operator T which commutes with its adjoint T ∗, that is, for a
normal operator on V . However, we shall stop here! Finally!
THE END
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
The following books can be used as references.
(1) Advanced Engineering Mathematics (Tenth Edition)
written by Erwin Kreyszig, and published by
John Wiley & Sons, Hoboken, N.J. (2011).
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
The following books can be used as references.
(1) Advanced Engineering Mathematics (Tenth Edition)
written by Erwin Kreyszig, and published by
John Wiley & Sons, Hoboken, N.J. (2011).
(2) Elementary Linear Algebra with Applications
(Tenth Edition) written by H. Anton, and published by
John Wiley & Sons, Hoboken, N.J. (2010).
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
The following books can be used as references.
(1) Advanced Engineering Mathematics (Tenth Edition)
written by Erwin Kreyszig, and published by
John Wiley & Sons, Hoboken, N.J. (2011).
(2) Elementary Linear Algebra with Applications
(Tenth Edition) written by H. Anton, and published by
John Wiley & Sons, Hoboken, N.J. (2010).
Acknowledgement
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
The following books can be used as references.
(1) Advanced Engineering Mathematics (Tenth Edition)
written by Erwin Kreyszig, and published by
John Wiley & Sons, Hoboken, N.J. (2011).
(2) Elementary Linear Algebra with Applications
(Tenth Edition) written by H. Anton, and published by
John Wiley & Sons, Hoboken, N.J. (2010).
Acknowledgement
I have made use of the lecture notes of similar courses
prepared by Murali Srinivasan and Jugal Verma (Spring
Semester of 2013 – 2014) and Akhil Ranjan (Spring Semester
of 2016 – 2017) at the Indian Institute of Technology Bombay.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21
A Chinese Poem
There once lived a man
who wanted to learn how to kill dragons.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
B.V. Limaye, IITDH MA 106: Lec-21