Algebra

Dr. Nguyen Tuan Long

Faculty of Economic Mathematics

Vector space

1. Euclidean vector space

2. Linear independence and Spanning set

3. Subspaces and dimension

4. General vector space

2/37
Euclidean vector space
Euclidean n-space
2-space= R^2

Definition
Euclidean n-space, also called Rn , is formally written as

Rn = {(x1 , x2 , . . . , xn ) | xi ∈ R}

We call x = (x1 , x2 , . . . , xn ) a vector in Rn , xi the i-th coordinate of the vector, and n

the dimension of the space. toạ độ

Examples
• (1, 2) ∈ R2 , but (1, i) ̸∈ R2 , where i2 = 1.
• (−2, 1, 0) ∈ R3 .
√
• (−1, 0, 2, π, 105 ) ∈ R5 .
3/37
Vector Operations

Let x = (x1 , x2 , . . . , xn ) and y = (y1 , y2 , . . . , yn ) be vectors in Rn and r ∈ R.

• Equality:
(x1 , x2 , . . . , xn ) = (y1 , y2 , . . . , yn ) means xi = yi for ∀ i.
• Addition:
(x1 , x2 , . . . , xn ) + (y1 , y2 , . . . , yn ) = (x1 + y1 , x2 + y2 , . . . , xn + yn ).
• Scalar multiplication:
r(x1 , x2 , . . . , xn ) = (rx1 , rx2 , . . . , rxn ).

Note: If we look at the vectors as column vectors, then equality, addition and scalar
multiplication are the same as matrix equality, addition and scalar multiplication.
4/37
Properties of vectors

Let x = (x1 , . . . , xn ), y = (y1 , . . . , yn ) and z = (z1 , . . . , zn ) be vectors in Rn and r, s ∈ R

be scalars. In addition, let 0 = (0, . . . , 0) and 1 = (1, . . . , 1), and −x = (−x1 , . . . , −xn ).
Addition Multiplication
(A1) x + y ∈ Rn (M1) rx ∈ Rn
(A2) x + y = y + x (M2) r(x + y) = rx + ry
(A3) x + 0 = x (M3) (r + s)x = rx + sx
(A4) x + (−x) = 0 (M4) (rs)x = r(sx)
(A5) x + (y + z) = (x + y) + z (M5) 1x = x

5/37
The dot product of vectors in Rn

Definition
Let x = (x1 , x2 , . . . , xn ), y = (y1 , y2 , . . . , yn ) be vectors in Rn , then the dot product (or
inner product) of x and y is

x · y = x1 y1 + x2 y2 + · · · + xn yn .

If x and y are column vectors (represented as n × 1 matrices), then

   
x1 y1
x2   y2 
   
x =  . , y =  . , x · y = x⊤ y = y⊤ x.
 ..   .. 
xn yn
6/37
The length of a vector in Rn

Definition
√
• in R2 : ||x|| = x21 + x22
√
• in R3 : ||x|| = x21 + x22 + x23
√ √
• in Rn : ||x|| = x21 + x22 + · · · + x2n = x · x ≥ 0

Property

1. ||rx|| = |r| ||x||

2. |x · y| ≤ ||x|| ||y|| (Cauchy-Schwarz inequality)
3. ||x + y||2 + ||x − y||2 = 2(||x||2 + ||y||2 )) (Parallelogram equality)
4. ||x + y|| ≤ ||x|| + ||y|| (Triangle inequality)
7/37
The angle between vectors in Rn
góc

Definition
Let x, y be nonzero vectors in Rn , then the angle θ between x and y is determined by
x·y
cos θ = .
||x|| ||y||

8/37
The distance between two vectors in Rn

Euclidean distance
Let x, y be vectors in Rn , then the Euclidean distance between x and y is

de (x, y) = ||x − y||.

Property

1. de (x, y) ≥ 0 with equality if and only if x = y

2. de (x, y) = de (y, x)
3. de (x, z) ≤ de (x, y) + de (y, z) (Triangle inequality)

9/37
The distance between two vectors in Rn
ấ
tính ch t t ương đồng
Cosine similarity and cosine distance
Let x, y be vectors in Rn , then the cosine similarity between x and y is

sc (x, y) = cos θ.

and the cosine distance between x and y is

dc (x, y) = 1 − cos θ.

10/37
Cosine Similarity

11/37
Cosine Similarity vs. Euclidean Distance

dc (OA,OC) = 1- cos(OA, OB)= 0.387

sc(OA, OC) = 0.613

euclidean distance= 7.9999

12/37
What are the purposes of using vector spaces

Vector space model are to consider the relationship between data that are
represented by vectors.
• Each object in the dataset is represented as a vector of features.
• The dimension of a vector is the number of features.

13/37
Cosine similarity

Model cars as a 3-dimensional space

Every car is expressed as a vector x = (x1 , x2 , x3 ), where
• x1 represents max_speed (maximum speed in kmph)
• x2 represents acceleration_time (number seconds to reach 100 kmph)
• x3 represents price

Example:
• porsche = (304, 4.4, 99200) porsche tesla bmw
• tesla = (250, 3.8, 79990) porsche 1.00 0.43 -0.93
• bmw = (160, 7, 60000) tesla 0.43 1.00 -0.73
bmw -0.93 -0.73 1.00
14/37
Cosine similarity

x = “That is a very happy person” =⇒ (1, 1, 1, 1, 1, 1, 0, 0, 0, 0)

y = “That is a happy person” =⇒ (1, 1, 1, 0, 1, 1, 0, 0, 0, 0)
z = “That is a happy dog” =⇒ (1, 1, 1, 0, 1, 0, 1, 0, 0, 0)
t = “Today is a sunny day” =⇒ (0, 1, 1, 0, 0, 0, 0, 1, 1, 1)

That is a very happry person dog today sunny day

x 1 1 1 1 1 1 0 0 0 0
y 1 1 1 0 1 1 0 0 0 0
z 1 1 1 0 1 0 1 0 0 0
t 0 1 1 0 0 0 0 1 1 1

15/37
Cosine similarity

ỏ
góc nh => t ương đồng lớn

16/37
Cosine distance and Euclidean distance

[Link]
17/37
Cosine distance and Euclidean distance

Sorted by Euclidean distance Sorted by cosine distance

Euclidean consine Euclidean consine

Australia 0.0000e+00 -2.2204e-16 Australia 0.0000e+00 -2.2204e-16
Mexico 1.5338e+11 7.9495e-03 Colombia 8.9811e+11 1.7206e-03
Spain 3.4119e+11 3.0579e-03 Cuba 1.1260e+12 2.4839e-03
Turkey 3.7982e+11 3.5028e-03 Italy 1.0883e+12 2.6777e-03
Indonesia 4.0835e+11 7.4176e-03 Argentina 7.5723e+11 2.9301e-03

Compare between Australia to all other countries on the list based on the
selected economic metrics
[Link]

18/37
Vector embedding space

19/37
Further applications

• Document Similarity.
• Recommendation Systems.
• Text Classification.
• Information Retrieval.
• Clustering.

20/37
Linear independence and Spanning set
Linear combination of vectors

Definition
If x1 , x2 , . . . , xm are vectors in Rn , then the vector

y = r1 x1 + r2 x2 · · · + rm xm

is called a linear combination of x1 , x2 , . . . , xm .

Examples
• (4, 2) is a linear combination of (1, 1) and (2, 0). conbination of
• [ ][ ] [ ] [ ] [ ] [ ] (1,2), (3,2)
1 3 x1 4 1 3 4
= =⇒ x1 + x2 = .
2 2 x2 0 2 2 0
21/37
Linear independence

Definition
The set of vectors P = {x1 , x2 , . . . , xm } ⊆ Rn is linearly independent if

0 = r1 x1 + r2 x2 · · · + rm xm (⋆)

then it must be that r1 = r2 = . . . = rm = 0. If there is at least one ri ̸= 0 so that (⋆) is

satisfied then the set P is called linearly dependent.

Examples
Verify that
• P = {x1 = (1, 2, 0), x2 = (0, 1, 1), x3 = (−1, 0, 2)} is linearly independent.
• P ∪ {x4 = (1, 1, 1)} is linearly dependent.
22/37
Linear independence in R2

• P = {x1 } ̸= {0} is always linearly independent.

• P = {x1 , x2 } is either linearly dependent or independent.
• P = {x1 , x2 , x3 } is always linearly dependent.
• Every set of more than 2 vectors is always linearly dependent.

23/37
Finding a maximum linearly independent set
ấ
duy nh t..

Problem: given P = {x1 , x2 , . . . , xm } ⊆ R , find a subset S of P of maximum size

n

such that S is linearly independent.

Hint. Consider a matrix A that has x1 , x2 , . . . , xm as its rows, and reduce A to an

echelon matrix. Then, all the nonzero rows form a maximum subset of linearly
independent vectors.

24/37
Linear independence

Property
• If P = {x1 , x2 , . . . , xm } ⊆ Rn is linearly independent then m ≤ n.
• Every set P = {x1 , x2 , . . . , xm } ⊆ Rn with m > n is linearly dependent.

Proof
Hint. Write r1 x1 + r2 x2 · · · + rm xm = 0 ⇐⇒ Ar = 0, the system always has nontrivial
solutions as m > n.

25/37
Spanning sets

Definition
For a set of vectors P = {x1 , x2 , . . . , xm } ⊆ Rn , we denote by Span(P) the set of all
linear combination of vectors in P. That is,

Span(P) = {y | y = r1 x1 + r2 x2 · · · + rm xm , ri ∈ R}.

Spanning sets in R2
• Span({x = (a, b)}) = ?
• Span({x1 , x2 }) = ? when P is linearly independent/dependent?
• Span(P) when P has more than 3 vectors? s ố ớ ơ
vector l n h n s ố ề
chi u m>n =>

dependen nên span P v n là R^2 ẫ

26/37
Spanning sets

Property

Span(P) = Span(S),

where S ⊆ P is a subset of maximum size and is linearly independent.

Proof
Suppose P is linearly dependent, then there is ri ̸= 0 such that

−rk
xi = r′1 x1 + · · · + ri′−1 xi−1 + r′i+1 xi+1 + · · · + r′m xm , where r′k = .
ri

Remove xi from P and repeat the process until the remaining vectors is linear
independent, and we obtain a desired set S. 27/37
Spanning sets

Spanning vectors
A set of vectors P = {x1 , x2 , . . . , xm } ⊆ Rn is spans Rn if every y ∈ Rn is a linear
combination of vectors in P, i.e., there exist r1 , r2 , . . . , rm so that

y = r1 x1 + r2 x2 · · · + rm xm .

We denote Rn = Span(P).
bình thường sẽ ậ
là t p con.....

Examples
Verify that
• P = {x1 = (1, 2, 0), x2 = (0, 1, 1)} does not span R3 . span p chỉ là tập con của R3
• P = {x1 = (1, 2, 0), x2 = (0, 1, 1), x3 = (−1, 0, 2)} spans R3 .
28/37
Bases of Rn

Definition
If a set P = {x1 , x2 , . . . , xm } ⊆ Rn is called a basic of Rn if it spans Rn and m = n.

Examples
• P = {x1 = (1, 2, 0), x2 = (0, 1, 1), x3 = (−1, 0, 2)} is a basic of R3 .
• P = {x1 = (1, 0, 0), x2 = (0, 1, 0), x3 = (0, 0, 1)} is a standard basic of R3 .

29/37
Bases of Rn

Properties
• If P is a basic of Rn then P is linearly independent
• Every basic of Rn has exactly n vectors
• There are infinitely many bases of Rn
• dim(Rn ) = n

30/37
Subspaces and dimension
Subspaces of Rn

Definition
A nonempty subset S ⊆ Rn is a subspace of Rn if it satisfies the following conditions
• If x, y ∈ S then x + y ∈ S
• If x ∈ S and r ∈ R then rx ∈ S

Example
• {0} is zero subspace of Rn
• {(x, 0) | x ∈ R} and {(0, y) | y ∈ R} are a subspaces of R2
• {(x, y) | x − y = 0, x, y ∈ R} is a subspace of R2
• {(x, y) | x + y = 1, x, y ∈ R} is not a subspace of R2
• For any nonempty set of vectors P, Span(P) is a subspace of Rn
31/37
Examples of Subspaces of Rn

Subspaces in R2 Subspaces in R3

32/37
Bases and dimension of subspaces

Definition
Let S be a subspace of Rn . A set P = {x1 , x2 , . . . , xm } ⊆ S is a basic of S if
• If P is linear independent 1 vec tor :ktra xem có khác vector 0 hay ko

• Span(P) = S 2 vector: xem có cùng ph ương ko

and m is called the dimension of the subspace S.

Example
• The zero space has no basic and has a dimension of 0
• (1, 0) is a basic of the subspace Span({(1, 0)})
• Find a basic of the subspace {(x, y) | x − y = 0, x, y ∈ R}

33/37
Hyperplanes as subspaces in Rn
ẳ
sieeu ph ng

Hyperplanes
∑
For a given a number b ∈ R and a tuple (r1 , r2 , . . . , rn ) ∈ Rn , where i r2i ̸= 0, the
subset S ⊆ Rn containing all the vectors (x1 , x2 , . . . , xn ) ∈ Rn such that

r1 x1 + r2 x2 + · · · + rn xn = b
n=2 => đ ường thẳng
is called a hyperplane. ặ
n=3=> m t ph ng ẳ

Hyperplanes as subspaces
A hyperplane H is a subspace of Rn if and only if it passes through the origin 0. In this
case the dimension of H is n − 1. khi qua gốc toạ độ , chiều = n-1...

34/37
Properties of subspaces

Properties
Let S1 , S2 are subspaces of Rn .
• S1 ∩ S2 = {x | x ∈ S1 , x ∈ S2 } is a subspace of both S1 , S2
• S1 + S2 = {x + y | x ∈ S1 , y ∈ S2 } is a subspace of Rn

dim(S1 + S2 ) = dim(S1 ) + dim(S2 ) − dim(S1 ∩ S2 )

• S1 ∪ S2 = {x | x ∈ S1 or x ∈ S2 } may not a subspace of Rn

Example
Find a basic of the subspace which is the intersection of two hyperplanes in R3 :

(H1 ) : 2x1 + x2 − x3 = 0 and (H2 ) : x1 + 3x2 + x3 = 0. 35/37

General vector space
General vector space

A vector space V is a non-empty set equipped with an addition operation and a

scalar multiplication operation such that for all x, y, z ∈ V and for all r, s ∈ R:
Addition Multiplication
(A1) x + y ∈ V (M1) rx ∈ V
(A2) x + y = y + x (M2) r(x + y) = rx + ry
(A3) ∃ 0 ∈ V : x + 0 = x (M3) (r + s)x = rx + sx
(A4) ∃ − x ∈ V : x + (−x) = 0 (M4) (rs)x = r(sx)
(A5) x + (y + z) = (x + y) + z (M5) 1x = x

Think of vectors in V as m × n matrices, or polynomials of degree of at most n!

36/37
General vector space

Example
• The set Rn×1 of all column matrices is a vector space
• The set R1×n of all row matrices is a vector space
• The set Rm×n of all m × n matrices is a vector space
• The set Pn of all polynomials of degree at most n with real coefficients is a
vector space with the foregoing addition and scalar multiplication

37/37