Dot product and inner product
Zdeněk Dvořák
February 24, 2015
1 Dot (scalar) product of real vectors
Definition 1. Let u = (α1 , . . . , αn ) and v = (β1 , . . . , βn ) be vectors from Rn .
The dot product of u and v is
u · v = α1 β1 + α2 β2 + . . . + αn βn .
Definition 2. The Euclidean norm of v = (α1 , . . . , αn ) ∈ Rn is
q √
|v| = α12 + α22 + . . . + αn2 = v · v.
θ u
Lemma 1 (Geometric interpretation). For any u, v ∈ Rn such that the angle
between u and v is θ,
u · v = |u||v| cos θ.
Proof. Note that the dot product is commutative and linear in both argu-
ments, and thus
(u − v) · (u − v) = u · (u − v) − v · (u − v)
= (u · u − u · v) − (v · u − v · v)
= u · u + v · v − 2u · v
Recall that in a triangle
1
v
c u−v
b
θ θ u
a
we have
c2 = a2 + b2 − 2ab cos θ,
and thus
|u − v|2 = |u|2 + |v|2 − 2|u||v| cos θ.
It follows that
|u|2 + |v|2 − |u − v|2
|u||v| cos θ =
2
u · u + v · v − (u − v) · (u − v)
=
2
u · u + v · v − (u · u + v · v − 2u · v)
=
2
2u · v
=
2
= u · v.
Uses of dot product:
• Determining the angle between two vectors:
θ u
u·v
θ = arccos
|u||v|
• Two vectors are perpendicular iff their dot product is 0.
2
• Orthogonal projection:
v
θ u
The projection p of u on v (where θ is the angle between u and v) has
norm
u·v
|u| cos θ =
|v|
and the same direction as v, hence
v u·v u·v
p= · = v.
|v| |v| v·v
• Determining coordinates in an orthogonal basis (projections to basis
vectors).
√ √ √ √
Example 1. Let u1 = ( 2/2, 2/2) and u2 = (− 2/2, 2/2). Determine
the coordinates of (3, 5) with respect to the basis B = u1 , u2 .
(3, 5)
u2 u1
Note that u1 · u2 = 0 (the vectors u1 and u2 are perpendicular) and |u1 | =
|u2 | = 1. Hence, the coordinates are
√
(3, 5) · u1 = 4 2
3
and √
(3, 5) · u2 = 2
2 Inner product spaces
Recall:
• R: the field of real numbers
• C: the field of complex numbers
• complex conjugation:
– α + βi = α − βi
– x+y =x+y
– xy = x y
p
– xx = |x|2 , where |α + βi| = α2 + β 2
Definition 3. Let F be either R or C.
Inner product space is a vector space V over F, together with an inner
product
h·, ·i : V2 → F
satisfying the following axioms:
positive definiteness For all v ∈ V, hv, vi is a non-negative real number,
and hv, vi = 0 if and only if v = o.
linearity in the first argument For all u, v, w ∈ V and α ∈ F,
hu + v, wi = hu, wi + hv, wi
hαu, wi = α hu, wi
conjugate commutativity For all u, v ∈ V,
hu, vi = hv, ui.
Remark:
• ho, vi = 0 = hv, oi for every v ∈ V.
• If F = R, then
4
– the last axiom states commutativity hu, vi = hv, ui, and
– h·, ·i is linear in the second argument as well
hw, u + vi = hw, ui + hw, vi
hw, αui = α hw, ui
• If F = C, then
hw, u + vi = hu + v, wi
= hu, wi + hv, wi
= hu, wi + hv, wi
= hw, ui + hw, vi
hw, αui = hαu, wi
= α hu, wi
= αhu, wi
= α hw, ui .
– h·, ·i is not linear in the second argument, because of the conjuga-
tion in scalar multiplication.
Example 2.
• Dot product gives an inner product on Rn .
• Another example of possible inner product on R2 :
h(α1 , α2 ), (β1 , β2 )i = 2α1 β1 + α2 β2 − α1 β2 − α2 β1
– positive definiteness: h(α1 , α2 ), (α1 , α2 )i = α12 + (α1 − α2 )2 ≥ 0,
and equal to 0 if and only if α1 = 0 and α1 − α2 = 0 ⇒ α2 = 0.
• Complex dot product on Cn :
(α1 , . . . , αn ) · (β1 , . . . , βn ) = α1 β1 + . . . + αn βn .
• Standard inner product on the space of continuous functions f : [α, β] →
R: Z β
hf, gi = f (x)g(x)dx
α
5
Definition 4. Let V be an inner product space. Vectors u, v ∈ V are
orthogonal if hu, vi = 0. We write u ⊥ v.
Example 3.
• (1, 0) and (0, 1) are orthogonal with respect to the dot product, since
(1, 0) · (0, 1) = 0.
• (1, 0) and (0, 1) are not orthogonal with respect to the inner product
h(α1 , α2 ), (β1 , β2 )i = 2α1 β1 + α2 β2 − α1 β2 − α2 β1 ,
since h(1, 0), (0, 1)i = −1.
• f (x) = sin x and g(x) = 1 are orthogonal with respect to the standard
inner product on the space of continuous functions from [0, 2π]:
Z 2π Z 2π
f (x)g(x)dx = sin x dx = [− cos x]2π
0 = cos 0 − cos(2π) = 0.
0 0
Theorem 2 (Pythagoras theorem). Let V be an inner product space and let
u, v ∈ V. If u ⊥ v, then
hu, ui + hv, vi = hu + v, u + vi .
Proof.
hu + v, u + vi = hu, ui + hv, vi + hu, vi + hv, ui
= hu, ui + hv, vi ,
since hu, vi = 0 and hv, ui = hu, vi = 0.
Theorem 3 (Cauchy-Schwarz inequality). Let V be an inner product space.
Then for all u, v ∈ V,
| hu, vi |2 ≤ hu, ui hv, vi ,
and if u and v are linearly independent, then the inequality is sharp.
Proof. The claim is clearly true if v = o, hence assume that hv, vi > 0.
6
v
w z
u
hu,vi
Let w = hv,vi
v and z = u − w. Then
hu, vi
hz, vi = u − v, v
hv, vi
hu, vi
= hu, vi − v, v
hv, vi
hu, vi
= hu, vi − hv, vi
hv, vi
= 0,
and thus v ⊥ z and w ⊥ z.
Since u = w + z, Pythagoras theorem implies
hu, ui = hw, wi + hz, zi
≥ hw, wi
hu, vi hu, vi
= hv, vi
hv, vi hv, vi
hu, vi 2
= hv, vi
hv, vi
| hu, vi |2
= ,
hv, vi
and thus
hu, ui hv, vi ≥ | hu, vi |2 .
hu,vi
The equality holds only if z = o, i.e., if u = hv,vi
v, which implies that u and
v are linearly dependent.
Example 4. Let x1 , . . . , xn be positive real numbers. Prove that
(x1 + . . . + xn )2
x21 + ... + x2n ≥ ,
n
7
where the equality holds if and only if x1 = x2 = . . . = xn .
Proof. We apply the Cauchy-Schwarz inequality for the dot product of u =
(x1 , . . . , xn ) and v = (1, . . . , 1):
(x21 + . . . + x2n )n = (u · u)(v · v)
≥ (u · v)2
= (x1 + . . . + xn )2 ,
where the equality only holds if u and v are linearly dependent, i.e., x1 =
. . . = xn .
Definition 5. Let V be a vector space over a field F ∈ {R, C}. A function
s : V → R is a norm if
• s(v) ≥ 0 for every v ∈ V, and s(v) = 0 if and only if v = o.
• s(αv) = |α|s(v) for every v ∈ V and α ∈ F.
• s(u + v) ≤ s(u) + s(v) for every u, v ∈ V (triangle inequality).
Definition 6. The norm induced by an inner product is
p
kvk = hv, vi.
• If h·, ·i is the dot product, then k · k is the Euclidean norm.
• Pythagoras theorem reformulated using the norm: if u ⊥ v, then
kuk2 + kvk2 = ku + vk2
• Cauchy-Schwarz inequality reformulated using the norm:
| hu, vi | ≤ kukkvk
• The triangle inequality holds because of Cauchy-Schwarz:
ku + vk2 = hu + v, u + vi
= kuk2 + kvk2 + hu, vi + hv, ui
≤ kuk2 + kvk2 + 2| hu, vi |
≤ kuk2 + kvk2 + 2kukkvk
= (kuk + kvk)2
