Inner-product spaces

In this topic, we introduce one of the fundamental concepts in mathematics, the inner-product
spaces.
Definition
Let V be a real vector space. An inner-product on V is a function h−, −i : V × V → R
satisfying the following axioms, for every u, v , w ∈ V and λ ∈ R.
(i) hu, v i = hv , ui (symmetry property)
(ii) hu + v , w i = hu, w i + hv , w i (additivity axiom)
(iii) hλu, v i = λhu, v i
(iv) hu, ui ≥ 0 and hu, ui = 0 if and only if u = 0 (positivity axiom)

An inner-product space is a pair (V , h−, −i) where V is a vector space and h−, −i is an
inner-product on V .
We will use h|i to denote the inner-product between the vectors u, v ∈ V .
() November 2, 2021 1 / 17
Example 1
Let V = Rn and define h|i : V × V → R by; where u, v ∈ Rn , u = (u1 , . . . , un ) and
v = (v1 , . . . , vn ) by
n
X
hu|v i = u · v = u1 v1 + · · · + un vn = ui v i .
i=1

Then (V , h|i) is an inner-product space.

Proof.
Let u, v , w ∈ Rn such that u = (u1 , . . . , un ), v = (v1 , . . . , vn ) and w = (w1 , . . . , wn ) and let
λ ∈ R. We need to prove that h|i satisfies axioms (i) to (iv).
(i)
X n Xn
hu|v i = u · v = ui vi = vi ui = v · u = hv |ui.
i=1 i=1

() November 2, 2021 2 / 17
(ii)
n
X
hu + v |w i = (ui + vi )wi
i=1
Xn
= (ui wi + vi wi )
i=1
n
X n
X
= ui wi + vi wi
i=1 i=1
= hu|w i + hv |w i.

(iii)
n
X n
X
hλu|v i = (λui )vi = λ ui vi = λhu|v i.
i=1 i=1

() November 2, 2021 3 / 17
 n
X
(iv) hu|ui = ui2 ≥ 0 since ui2 ≥ 0 for all i = 1, . . . , n.
i=1
n
X
Also hu|ui = 0 implies ui2 = 0, which then implies ui2 = 0. This then implies that ui = 0
i=1
for all i = 1, . . . , n.

() November 2, 2021 4 / 17
Example 2
Let V = C [a, b] denote the vector space of continous real-valued functions defined on the
closed and bounded interval [a, b]. Define h|i : V × V → R by
Z b
hf |g i = f (x)g (x)dx; f , g ∈ V . Prove that (V , h|i) is an inner-product space.
a

Proof.
Z b Z b
(i) hf |g i = f (x)g (x)dx = g (x)f (x)dx = hg |f i.
a a
(ii)
Z b
hf + g |hi = [(f + g )(x)h(x)]dx
a
Z b
= ([f (x) + g (x)]h(x))dx
a

() November 2, 2021 5 / 17
Example 2 cont’d

Z b
= (f (x)h(x) + g (x)h(x))dx
a
Z b Z b
= f (x)h(x)dx + g (x)h(x)dx
a a
= hf |hi + hg |hi
(iii)
Z b
hλf |g i = (λf )(x)g (x)dx
a
Z b
= λf (x)g (x)dx
a
Z b
=λ f (x)g (x)dx = λhf |g i
a
() November 2, 2021 6 / 17
 Z b
(iv) hf |f i = [f (x)]2 dx ≥ 0 since [f (x)]2 ≥ 0 for every x ∈ [a, b].
a Z b
If hf |f i = 0, then [f (x)]2 dx = 0. Since [f (x)]2 ≥ 0 for all x ∈ [a, b] and f is continuous,
Z b a
2
then [f (x)] dx = 0 which implies that f (x) = 0 for all x ∈ [a, b].
a
Conversely, if f = 0, then f (x) = 0 for all x ∈ [a, b] and hence
Z b Z b
hf |f i = [f (x)]2 dx = 0dx = 0.
a a

() November 2, 2021 7 / 17
Norm and Distance in Inner-product spaces

From axiom 4 in the definition of the inner-product,

p we have that for every vector u in an
inner product space V , hu|ui ≥ 0, and therefore hu|ui exists.

Definition
Let u ∈ V be any vector in an inner-product space V . The length (norm) of u, denoted by
||u|| is defined as p
||u|| = hu|ui.

Definition
The distance between two vectors u, v ∈ V in an inner-product space V is given by
p
d(u, v ) = ||u − v || = hu − v |u − v i.

() November 2, 2021 8 / 17
Example
Let V = P2 be the vector space of polynomials of degree ≤ 2 with inner product defined by
Z 1
hp|qi = p(x)q(x)dx.
−1

Let p(x) = 1 and q(x) = x. Find

(i) ||p||
(ii) ||q||
(iii) d(p, q).
Solution:
(i) sZ sZ
p 1 1 q √
||p|| = hp|pi = p(x)p(x)dx = 12 dx = [x]1−1 = 2.
−1 −1

() November 2, 2021 9 / 17
Example cont’d
(ii) sZ sZ s
1 1 1 r
p x3 2
||q|| = hq|qi = q(x)q(x)dx = x 2 dx = = .
−1 −1 3 −1 3
(iii)
p
d(p, q) = ||p − q|| = hp − q|p − qi
sZ
1
= [p(x) − q(x)]2 dx
−1
sZ
1
= (1 − x)2 dx
−1
r
8
= .
3
() November 2, 2021 10 / 17
Exercise!

Let P2 be the vector space of polynomials of degree ≤ 2 and define the inner-product by

hp|qi = a0 b0 + a1 b1 + a2 b2

where p(x) = a0 + a1 x + a2 x 2 , q(x) = b0 + b1 x + b2 x 2 . Let p(x) = −2 + x + 3x 2 and

q(x) = 4 − 7x 2 . Compute
(i) hp|qi
(ii) ||q||
(iii) ||p||
(iv) d(p, q)

() November 2, 2021 11 / 17
Cauchy-Schwartz Inequality
Theorem
Let u, v ∈ V be vectors in an inner-product space V . Then |hu|v i| ≤ ||u||||v ||.

Proof.
If u = 0, the inequality holds trivially. So suppose u 6= 0 and let a = hu|ui, b = 2hu|v i and
c = hv |v i.
Consider htu + v |tu + v i for some t ∈ R. By the positivity axiom,

htu + v |tu + v i ≥ 0
htu|tu + v i + hv |tu + v i ≥ 0
2
t hu|ui + thu|v i + thv |ui + hv |v i ≥ 0
t 2 hu|ui + 2thu|v i + hv |v i ≥ 0
at 2 + bt + c ≥ 0
() November 2, 2021 12 / 17
 b 2 c
   2
b
t+ + − ≥0
2a a 2a

b 2
   2
c b
Since t + ≥ 0, it is enough to have − ≥ 0.
2a a 2a
 2
c b
So from − ≥ 0, we have 4ac − b 2 ≥ 0, which implies that b 2 ≤ 4ac.
a 2a
Using a = hu|ui, b = 2hu|v i and c = hv |v i, we have

4(hu|v i)2 ≤ 4hu|uihv |v i

(hu|v i)2 ≤ ||u||2 ||v ||2
|hu|v i| ≤ ||u||||v ||.

() November 2, 2021 13 / 17
Properties of length and distance
In the theorem below, we state the properties of length and distance.
Theorem
Let u and v be vectors in an inner-product space V . Then
(i) ||u|| ≥ 0 and ||u|| = 0 if and only if u = 0
(ii) ||ku|| = k||u||, for every constant k
(iii) ||u + v || ≤ ||u|| + ||v || ( triangle inequality)

Proof.
(i) This follows directly from axiom 4 of the definition of inner product and the definition of
norm. (Please show this students!)
p u ∈ V and
(ii) For any pfor any constant
p k, we have
||ku|| = hku|kui = k 2 hu|ui = k 2 ||u||2 = k||u||.

() November 2, 2021 14 / 17
(iii) For any vectors u, v ∈ V , we have

||u + v ||2 = hu + v |u + v i
= hu|ui + 2hu|v i + hv |v i
= ||u||2 + 2hu|v i + ||v ||2 .

From the Cauchy-Schwartz inequality, we have that hu|v i ≤ ||u| ||v ||.
Therefore,

||u + v ||2 ≤ ||u||2 + 2||u||||v || + ||v ||2

= (||u|| + ||v ||)2

Taking square root on bot sides yields ||u + v || ≤ ||u|| + ||v ||.

() November 2, 2021 15 / 17
Orthogonal vectors
Definition
Two vectors u, v ∈ V are said to be orthogonal if hu|v i = 0.

In the next theorem, we prove the generalized Pythagoras theorem for orthogonal vectors.
Theorem
Let u and v be orthogonal vectors in an inner product space V . Then
||u + v ||2 = ||u||2 + ||v ||2 .

Proof: Since u and v are orthogonal, then hu|v i = 0 and hence

||u + v ||2 = hu + v |u + v i
= hu|ui + 2hu|v i + hv |v i
= ||u||2 + 2hu|v i + ||v ||2 = ||u||2 + ||v ||2 .

() November 2, 2021 16 / 17
Exercise

Let V =RP2 be the vector space of polynomials of degree ≤ 2 with inner product defined by
1
hp|qi = −1 p(x)q(x)dx. Let p(x) = x and q(x) = x 2 .
(i) Verify that p and q are orthogonal.
r
2 2 16
(ii) Compute ||p + q||. (Answers: ||p||2 = , ||q||2 = and final answer is ).
3 5 15

() November 2, 2021 17 / 17