Lecture no 6: INNER PRODUCT SPACES. EUCLIDEAN SPACE.

ORTHOGONALITY 1
Up to this point, we have made qualitative deductions about vector spaces, but
we have not derived quantitative properties. An example of a quantitative property that
would almost certainly be of interest is the length of a vector.
In this chapter, we introduce one way of forming the product of vectors, called
the inner product, that associates a scalar with each pair of vectors. Among other things,
an inner product enables us to generalize the notion of length that occurs in Euclidean
spaces.
The terms “inner product” and “dot product” are often used synonymously, but
they are not exactly the same. The dot product is used only when the scalar field is ℝ.
The inner product generalizes the dot product and applies when the scalar field is ℝ or
ℂ.

1. Inner Product Spaces

Definition. Let V be a real vector space. Suppose to each pair of vectors u, v  V there
is assigned a real number, denoted by u, v. This function is called a (real) inner product
on V if it satisfies the following axioms:
[I1] (Linear Property): au1 + bu2, v = au1, v + bu2, v.
[I2] (Symmetric Property): u, v = v; u.
[I3] (Positive Definite Property): u, u  0.; and u, u = 0 if and only if u = 0.

The vector space V with an inner product is called a (real) inner product space.

Axiom [I1] states that an inner product function is linear in the first position.
Using [I1] and the symmetry axiom [I2], we obtain

That is, the inner product function is also linear in its second position. Combining these

9 Л 2 Евклідов простір. Аксіоми скалярного добутку. Існування

ортонормованого базису евклідового простору. Ортонормовані
системи векторів. Процес ортогоналізації Грама-Шмідта.
10 ПР 2 Побудова ортонормованого базису евклідового простору.
two properties and using induction yields the following general formula:

That is, an inner product of linear combinations of vectors is equal to a linear

combination of the inner products of the vectors.

Example Let V be a real inner product space. Then, by linearity

Remark Axiom [I1] by itself implies 0, 0 = 0v, 0 = 0v, 0 = 0. Thus, [I1], [I2], [I3]
are equivalent to [I1], [I2], and the following axiom:
[I’3] If u  0; then u, u is positive.

That is, a function satisfying [I1], [I2], [I’3] is an inner product.

Norm of a Vector
By the third axiom [I3] of an inner product, u, u is nonnegative for any vector u. Thus,
its positive square root exists. We use the notation
𝑢 = √〈𝑢, 𝑢〉

This nonnegative number is called the norm or length of u. The relation 𝑢2 = 〈𝑢, 𝑢〉
will be used frequently.

Remark If 𝑢 = 1 or, equivalently, if 〈𝑢, 𝑢〉 = 1 then u is called a unit vector and it
is said to be normalized. Every nonzero vector v in V can be multiplied by the reciprocal
of its length to obtain the unit vector
𝑣̂ = 𝑣⁄‖𝑣 ‖
which is a positive multiple of v. This process is called normalizing v.
Examples of Inner Product Spaces
1. Euclidean n-Space Rn
Consider the vector space Rn. The dot product or scalar product in Rn is defined by

where u = (ai) and v = (bi). This function defines an inner product on Rn. The norm 𝑢
of the vector u = (ai) in this space is as follows

On the other hand, by the Pythagorean theorem, the distance from the origin O in R3 to
a point P(a; b; c) is given by √𝑎2 + 𝑏 2 + 𝑐 2 . This is precisely the same as the above-
defined norm of the vector v = (a; b; c) in R3.

Because the Pythagorean theorem is a consequence of the axioms of Euclidean

geometry, the vector space Rn with the above inner product and norm is called Euclidean
n-space.

Although there are many ways to define an inner product on Rn, we shall assume this
inner product unless otherwise stated or implied. It is called the usual (or standard )
inner product on Rn.

Remark Frequently the vectors in Rn will be represented by column vectors—that is,

by n  1 column matrices. In such a case, the formula
u, v = uT v
defines the usual inner product on Rn.

Example Let u = (1; 3;-4; 2), v = (4;-2; 2; 1), w = (5;-1;-2; 6) in R4.

Function Space C[a; b] and Polynomial Space P(t)

The notation C[a; b] is used to denote the vector space of all continuous functions
on the closed interval [a; b]—that is, where a  t  b. The following defines an inner
product on C[a; b], where f (t) and g(t) are functions in C[a; b]:

It is called the usual inner product on C[a; b].

The vector space P(t) of all polynomials is a subspace of C[a; b] for any interval
[a; b], and hence, the above is also an inner product on P(t).

Example
Matrix Space M = Mm,n
Let M = Mm;n, the vector space of all real m  n matrices. An inner product is defined
on M by
A, B = tr (BT A),
where, as usual, tr () is the trace—the sum of the diagonal elements. If A = [aij] and
B = [bij], then
A, B = Tr (BT A) = ∑𝑚 𝑛
𝑖=1 ∑𝑗=1 𝑎𝑖𝑗 𝑏𝑖𝑗 , and

‖𝐴‖2 = 〈𝐴, 𝐴〉 = ∑𝑚 𝑛 2
𝑖=1 ∑𝑗=1 𝑎𝑖𝑗

That is, A, B is the sum of the products of the corresponding entries in A and B
and, in particular, A, A is the sum of the squares of the entries of A.

Hilbert Space
Let V be the vector space of all infinite sequences of real numbers (a1; a2; a3; ...)
satisfying

that is, the sum converges. Addition and scalar multiplication are defined in V
componentwise, that is, if

An inner product is defined in v by

u, v = a1b1 + a2b2 + a3b3 + ...

The above sum converges absolutely for any pair of points in V. Hence, the inner
product is well defined. This inner product space is called l2-space or Hilbert space.

2. Cauchy–Schwarz Inequality, Applications

The following formula is called the Cauchy–Schwarz inequality or Schwarz inequality.
It is used in many branches of mathematics.
Theorem : (Cauchy–Schwarz) For any vectors u and v in an inner product space V,

Next we examine this inequality in specific cases.

Example

The next theorem gives the basic properties of a norm. The proof of the third
property requires the Cauchy–Schwarz inequality.

Theorem : Let V be an inner product space. Then the norm in V satisfies the following
properties:

The property [N3] is called the triangle inequality, because if we view u + v as the side
of the triangle formed with sides u and v (as shown in Figure), then [N3] states that the
length of one side of a triangle cannot be greater than the sum of the lengths of the other
two sides.

Angle Between Vectors

For any nonzero vectors u and v in an inner product space V, the angle between u and v
is defined to be the angle  such that 0     and

By the Cauchy–Schwartz inequality, -1  cos   1, and so the angle exists and is unique.
Example
3. Orthogonality
Let V be an inner product space. The vectors u, v  V are said to be orthogonal
and u is said to be orthogonal to v if
u, v = 0
The relation is clearly symmetric—if u is orthogonal to v, then v, u = 0, and so v is
orthogonal to u. We note that 0  V is orthogonal to every v  V, because
0, v = 0v, v = 0v, v = 0.
Conversely, if u is orthogonal to every v  V, then u, u = 0 and hence u = 0 by
[I3]. Observe that u and v are orthogonal if and only if cos  = 0, where  is the angle
between u and v. Also, this is true if and only if u and v are “perpendicular”—that is, 
= /2 (or  = 90).

Example

Remark A vector w = (x1; x2; … ; xn) is orthogonal to u = (a1; a2; … ; an) in Rn if

 That is, w is orthogonal to u if w satisfies a homogeneous equation whose
coefficients are the elements of u.

Example Find a nonzero vector w that is orthogonal to u1 = (1; 2; 1) and u2 = (2; 5; 4)

in R3.

Any multiple of w will also be orthogonal to u1 and u2. Normalizing w, we obtain the
following unit vector orthogonal to u1 and u2:

Orthogonal Complements
Let S be a subset of an inner product space V. The orthogonal complement of S, denoted
by S (read “S perpendicular”) consists of those vectors in V that are orthogonal to every
vector u  S, that is,

In particular, for a given vector u in V, we have

that is, u consists of all vectors in V that are orthogonal to the given vector u.

We show that S is a subspace of V. Clearly 0  S, because 0 is orthogonal to

every vector in V. Now suppose v, w  S. Then, for any scalars a and b and any vector
u  S, we have

Thus, av + bw  S, and therefore S is a subspace of V.

We state this result formally: Let S be a subset of a vector space V. Then S is a

subspace of V.
Remark 1: Suppose u is a nonzero vector in R3. Then there is a geometrical description
of u. Specifically, u is the plane in R3 through the origin O and perpendicular to the
vector u. This is shown in Figure:

Remark 2: Let W be the solution space of an m  n homogeneous system AX = 0, where

A = [aij] and X = [xi]. Recall that W may be viewed as the kernel of the linear mapping
A: Rn  Rm. Now we can give another interpretation of W using the notion of
orthogonality.
Specifically, each solution vector w = (x1; x2; … ; xn) is orthogonal to each row of
A, hence, W is the orthogonal complement of the row space of A.

Example Find a basis for the subspace u of R3, where u = (1; 3;-4)

Suppose W is a subspace of V. Then both W and W are subspaces of V. The next

theorem, is a basic result in linear algebra.

Theorem : Let W be a subspace of V. Then V is the direct sum of W and W, that is,
𝑉 = 𝑊⨁𝑊 ⊥