Theoretical Models in

Computing and Applications

Nguyen V.M. Man,
Ph.D. in Statistics
15th September 2011
Email: [email protected] or
[email protected]
Contents
1 Basic Logic 5
1.1 Propositional logic . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.1 Theory- Content of Propositional Logic . . . . . . . . . . . 6
1.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.1.3 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . 14
1.1.4 Practical projects (Optional) . . . . . . . . . . . . . . . . . 14
1.2 Predicate logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.1 What topics will we learn? . . . . . . . . . . . . . . . . . . 17
1.2.2 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . 21
2 Set Theory 23
2.1 Sets- Ways of describing things . . . . . . . . . . . . . . . . . . . . 23
2.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.2 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.4 Key Mathematical Symbols . . . . . . . . . . . . . . . . . . 25
2.2 Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 Equivalence Relation . . . . . . . . . . . . . . . . . . . . . . 26
2.2.2 Partial orders . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.4 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . 27
2.2.5 Practical projects (Optional) . . . . . . . . . . . . . . . . . 28
2.3 Functions- Ways of tranforming things . . . . . . . . . . . . . . . . 29
2.3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.2 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.4 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . 30
2.3.5 Practical projects (Optional) . . . . . . . . . . . . . . . . . 31
2.4 Integers and Algorithms on Integers . . . . . . . . . . . . . . . . . 32
2.4.1 Review- Motivation . . . . . . . . . . . . . . . . . . . . . . 32
2.4.2 Introduction to Algorithms . . . . . . . . . . . . . . . . . . 33
2.4.3 Basic Integer Theory . . . . . . . . . . . . . . . . . . . . . . 33
3
4 CONTENTS
2.4.4 Application of Integers in Cryptology . . . . . . . . . . . . 36
2.4.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.6 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . 38
2.4.7 Practical projects (Optional) . . . . . . . . . . . . . . . . . 38
3 Counting Techniques. 39
3.1 Basic counting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.2 Theory on Countability . . . . . . . . . . . . . . . . . . . . 40
3.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Basic problems . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.2 Practical projects (Optional) . . . . . . . . . . . . . . . . . 42
4 Induction and Recursion 45
4.1 Inductively Dened Sets . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.1 How to do? . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2 Recursively Dened Functions and Procedures . . . . . . . . . . . 46
4.3 Inductive Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.4 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . . . . . 48
5 Graph Theory and Models 51
5.1 Undirected Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.1.1 Basic concepts and basic operations . . . . . . . . . . . . . 51
5.1.2 Simple Graphs and Connectivity . . . . . . . . . . . . . . . 53
5.1.3 Bipartite Graph . . . . . . . . . . . . . . . . . . . . . . . . 57
5.1.4 Graph Isomorphism . . . . . . . . . . . . . . . . . . . . . . 58
5.2 Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2.1 Binary Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.2.2 Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3 Digraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.3.1 Path Problem in Directed Graphs . . . . . . . . . . . . . . 62
5.3.2 Directed Graphs- The reachability matrix M

. . . . . . . . 63
5.4 Problems- Solving Hints . . . . . . . . . . . . . . . . . . . . . . . . 65
5.4.1 Problem 1: Basic concepts and operations . . . . . . . . . . 65
5.4.2 Problem 2: Representations of Graph Connectivity . . . . 65
5.4.3 Problem 3: Isomorphism Graph Invariants . . . . . . . . . 67
5.4.4 Problem 4: Bipartite graphs . . . . . . . . . . . . . . . . . . 69
5.4.5 Problem 5: Directed graphs . . . . . . . . . . . . . . . . . . 70
5.4.6 Problem 6: Trees . . . . . . . . . . . . . . . . . . . . . . . . 72
5.5 Algorithms on Graphs . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.5.1 Shortest paths and Dijkstra Algorithm . . . . . . . . . . . . 73
CONTENTS 1
5.5.2 Max-ow Min-cut algorithm by Ford-Fulkerson . . . . . . . 79
6 Algebraic Model and Polynomial Equations 81
6.1 Linear polynomial equations . . . . . . . . . . . . . . . . . . . . . . 81
6.1.1 Gauss Elimination - Contents . . . . . . . . . . . . . . . . . 82
6.1.2 Gauss Elimination- Technique . . . . . . . . . . . . . . . . . 87
6.1.3 Matlab section . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.1.4 Practical projects (Optional) . . . . . . . . . . . . . . . . . 91
6.2 Symbolic computation for nding roots of nonlinear equations . . . 91
6.3 Algebraic Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.3.1 In Industrial Statistics . . . . . . . . . . . . . . . . . . . . . 93
6.3.2 In Business Intelligence . . . . . . . . . . . . . . . . . . . . 99
7 Probabilistic Models 101
7.1 Practical Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 101
7.2 Concepts and Definitions . . . . . . . . . . . . . . . . . . . . . . . 103
7.2.1 Random Variable and Probability Distribution . . . . . . . 104
7.2.2 Why study Probability Distributions? . . . . . . . . . . . . 106
7.3 Discrete Probability Distributions . . . . . . . . . . . . . . . . . . . 107
7.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
2 CONTENTS
Introduction
This course provides a challenging introduction to some of the central ideas of
Theoretical Computer Science. It attempts to present a vision of computer
science beyond computers: that is, CS as a set of mathematical tools for
understanding complex systems such as universes and minds.
A. COURSE OVERVIEW: No. of Credits: 4
1. Course Objective: To provide undergraduate students with mathematical
reasoning, combinatorial analysis, and algorithmic thinking and modeling.
2. Prerequisite: None
3. Major Contents: Logic, Set, Algorithms, Mathematical Reasoning, Induction,
and Recursion, Counting, Graph Models and Algorithms on Graphs, Roots of
Polynomial Equations.
4. Assessment:
Assignments and Class Attendance: 30%
Midterm Test: 20%; Final Exam: 50%
5. Documents:
Main lecture note: yes
References:
a/ DISCRETE MATHEMATICS, Norman L. BIGGS, Oxford University Press,
Second Edition, 2009.
b/ ELEMENTARY METHODS IN NUMBER THEORY, Melvyn B.
Nathanson, Springer 2000
B. COURSE STRUCTURE
Part I: Discrete Structures, including logic, sets, counting techniques.
Part II provides Graph Models, together relavant structures.
Part III: (Chapter 7) Algebraic methods and modeling for nding roots
of equations, both linear and nonlinear equations.
Part IV (Chapter 8) Advanced: Probabilistic Modeling. How to deal
with uncertainty in daily life?
3
4 CONTENTS
C. Key remarks and Philosophy of Learning
I/ This subject covers many mathematical topics that are essential in Computer
Science but are not covered in the standard calculus curriculum. In addition,
the subject teaches students about careful mathematics:
- precisely stating assertions about well-dened mathematical objects and
- verifying these assertions using mathematically sound proofs.
II/ Students are encouraged to collaborate on problem sets (counting for 30% of
the nal grade) as on teams in class. Study groups can be a big help in
mastering course material, besides being fun and a good way to make friends.
However, students must write up solutions on their own,
neither copying solutions nor providing solutions to be copied.
The last page of each problem set has a collaboration statement to be
completed and attached as the rst page of a problem set submission:
I worked alone and only with course materials
or
I collaborated on this assignment with (students in class).

III/ Philosophy
1. Amount learned is proportional to time put in.
2. Best way to learn: to gure out ideas yourself or teach them to someone else.
3. Second best: to do so with hints from others (friends or us).
4. Third best: to get the ideas from reading; but pause in your reading to
think about them.
5. The object of a lecture: not so much to inform you of important facts, but
rather to stimulate you to try to learn about new concepts.
6. The object of the course: to empower you to use the concepts of Discrete
Mathematics and Theoretical Computation in any context.

Chapter 1
Basic Logic
We study logic from the mathematical point of view. This introductory chapter
is about mathematical logic, covers two key topics:
1. Propositions Propositional logic
2. Predicate logic
We begin the course by looking at examples involving everyday English
sentences. This is followed by an introduction to the more formal mathematical
approach used in propositional and predicate logic.
1.1 Propositional logic
Motivation
The fundamental objects we work with in arithmetic are numbers. In a similar
way, the fundamental objects in logic are propositions. Here are some examples
of English sentences that are propositions:
Canberra is the capital of Australia.
There are 8 days in a week.
Isaac Newton was born in 1642.
5 is greater than 7.
Every even number greater than 2 can be expressed as the sum of two
prime numbers.
Propositions truth values. The rst and third of these propositions are true,
and the second and fourth are false. It is not known at present whether the
fth proposition is true or false.
5
6 CHAPTER 1. BASIC LOGIC
The following sentences are not propositions:
Where are you going?
Come here!
How about these?
Anne is tall.
Ice cream is delicious.
x > 5.
The last of the three sentences given above is an example of a predicate.
1.1.1 Theory- Content of Propositional Logic
Propositions- Connectives and truth tables
Compound propositions- Writing compound propositions
Tautology Contradiction
Logical equivalence
Decomposing complicated proposition
Provable equivalence- Rule of Inference
Denition 1. A proposition is a statement that is either true or false.
Whichever of these (true or false) is the case is called the truth value of the
proposition.
a/ Connectives and truth tables
Logic is not concerned with determining the truth values of propositions of the
kind we have seen so far.
E.g. the truth value of Canberra is the capital of Australia
is a question of geography, not logic. The next example is dierent, however:
If Brian and Angela are not both happy, then either Brian is not happy or
Angela is not happy.
The sentence about Brian and Angela is an example of a compound proposition.
It is built up from the atomic propositions
Brian is happy, and Angela is happy
using the words and, or, not (also written ) and if-then. These words are
known as connectives. As we will see, the role of connectives in logic is
analogous to the role played by operations such as + and in algebra.
1.1. PROPOSITIONAL LOGIC 7
The study of the structure of compound propositions is made easier by the use
of symbols for atomic propositions and connectives.
We will use lower-case letters such as p, q and r to denote atomic
propositions. There are ve connectives that we will use in our work; they are
listed in Table 1.1.1, together with their symbols.
Connective Symbol
and
or
not
if-then
if-and-only-if
Table 1.1: Connectives and associated symbols
The propositional connectives are:
negation ( ),
conjunction
disjunction
implication or
biimplication
They are read as not, and, or, if-then, if and only if respectively.
The connectives , , , are designated as binary,
while is designated as unary.
r: the negation of r
p q: the conjunction of p and q
p q: ?
p q or p q: the implication of q by p
c d: logical equivalence
8 CHAPTER 1. BASIC LOGIC
Conventions

Let p, q be arbitrary statements. We notationally distinguish few cases:

The relationship between and
p q: the implication of q by p, or p implies q
p q: logical implication, p logically implies q.
We understand that:
if p q is a tautology, then we say
p logically implies q, and write p q.
The relationship between and
p q: bi-implication of p and q, p if and only if q
p q: logical equivalence, p is logically equivalent to q
We understand that:
if p q is a tautology, then we say
p is logically equivalent to q, and write p q.

b/ Compound propositions
With the exception of not, the symbols for these connectives are written
between the two operands (the propositions they connect).
For example, if
p denotes the proposition Today is Monday, and
q denotes the proposition It is raining,
then we can write
the symbol p, p or not p before the proposition to which it applies; thus,
p means Today is not Monday.
the symbol p q to denote the proposition
Today is Monday and it is raining.
Table 1.1.1 shows how to form the true value of p q
Can we form compound propositions p q, p q similarly?
1.1. PROPOSITIONAL LOGIC 9
p q p q
T T T
T F F
F T F
F F F
Table 1.2: Connective and
Notice that each column is obtained using the truth table for the principal
connective in the expression at the top of the column.
If an expression contains three variables (p, q and r, say), then the table
will have eight lines instead of four (there are 2
3
= 8 dierent ways of
allocating truth values to three expressions), but the method is the same.
Writing compound propositions in symbolic form
We now have the notation we need in order to be able to write compound
propositions in symbolic form. Express the proposition, e.g.
Either my program runs and it contains no bugs, or my program contains
bugs in symbolic form.
c/ Tautology Contradiction
Denition 2. You should know two following basic concepts to make right
reasoning.
- An expression that is always true, regardless of the truth values of the
variables it contains, is called a tautology.
- An expression that is always false, regardless of the truth values of the
variables it contains, is called a contradiction. Mathematically contradictions
are expressions of the form
or
where is any proposition (simple or compound).
Below examples shows how these concepts are used.
10 CHAPTER 1. BASIC LOGIC
1.1.2 Examples
Example 1. Express the proposition
Either my program runs and it contains no bugs,
or my program contains bugs in symbolic form.
Example 2. Construct the truth table for the expression (p q) q.
Example 3. Now look again at the proposition we introduced at the beginning
of this section
If Brian and Angela are not both happy,
then either Brian is not happy or Angela is not happy
a/ Write this compound proposition in symbolic form and
b/ provide its truth table.

d/ Logical equivalence
Denition 3. Two expressions (composed of the same variables) are logically
equivalent if they have the same truth values for every combination of the truth
values of the variables.
Informally, we could say that two expressions are logically equivalent if they
yield the same truth table. Formally,
A(p, q, r, ...) is logically equivalent to B(p, q, r, ...), write
A(p, q, r, ...) B(p, q, r, ...) if we have
A(p, q, ...) = B(p, q, ...) for every choice of boolean values of p, q, ...
Logical equivalence and connective if-and-only-if
There is a subtle but important distinction between the connective
if-and-only-if or and
the concept of logical equivalence .
1.1. PROPOSITIONAL LOGIC 11
When we write A B, we are writing a single logical expression.
(A B) (B A).
Logical equivalence, A B, on the other hand, is a relationship between two
logical expressions A and B. The connection of the two concepts is:
A and B are logically equivalent
if and only if the expression A B is a tautology.

e/ Converse and Contrapositive

Denition 4. We have
Expressions of the form p q are called implications.
The converse of p q is q p
The contrapositive of p q is q p
Example 4. Write down English sentences for the converse and the
contrapositive of:
If 250 is divisible by 4 then 250 is an even number.
Quiz 1. (5 min) Prove that
p q is not logically equivalent to its converse, but that it is logically
equivalent to its contrapositive. That is
p q q p

f/ Conditionals A B
Given that if A then B (or A implies B= A B) is a conditional
statement with hypothesis A and conclusion B.
Denition 5. Its contrapositive is: if not B then not A and
its converse is: if B then A.
Since statements with the same truth table are said to be equivalent; the table
shows that a conditional and its contrapositive are equivalent, we write
if A then B = if not B then not A
12 CHAPTER 1. BASIC LOGIC
Proof Techniques for P = Q. To prove a generic proposition P = Q, you
could use one of the following methods:
a direct argument or Conditional Proof:
to carry out the implication P = Q directly, know P is true.
a contrapositive argument: to use the identity
P = Q Q = P
(Prove this equality!)
a contradiction argument means that, provided P is true,
we assume Q is false and try to derive a contradiction
P Q = false
Proof By Contradiction
A false statement is called a contradiction.
For example, S and not S is a contradiction for any statement S.
A truth table will show us that
if A then B, is equivalent to [ A and (not B) ] implies false.
or in mathematical symbols
A B [(A notB) false]
So to prove if A then B, it suces
to assume A and also
to assume not B, and then
to argue toward a false statement.
This technique is called proof by contradiction.

1.1. PROPOSITIONAL LOGIC 13

g/ Decomposing complicated proposition

We began this section with an example of a complicated proposition that we

showed to be logically equivalent to a simpler one.
Occasions often arise in practice where it is desirable to replace a logical
expression with a simpler expression that is logically equivalent to it.
For example, we have seen how logical expressions representing propositions can
occur in algorithms and computer programs. By writing these expressions as
simply as possible, we can make a program more ecient and reduce the
chance of error.
How do we do?
In order to be able to simplify logical expressions eectively, we need to
establish a list of pairs of expressions that are logically equivalent.
We will use the symbol or placed between two expressions to indicate that
they are equivalent. A statement of the form
P Q
or
P Q
where P and Q are logical expressions is called a law of logic.
A list of the most important laws of logic is given as follows.
De Morgan (p q) p q
Contrapositive p q q p
Equivalence law p q (p q) (q p)
Implication law p q p q
Double negation law (p) p
Idempotent laws p p p
p p p
Table 1.3: Most important logic laws
14 CHAPTER 1. BASIC LOGIC
1.1.3 Problems- Solving Hints
1. Using important laws to simplify logical expressions
Quiz a. (10 min) Use the laws of logic to simplify the expression:
p (p q)
Quiz b. (10 min)
An algorithm contains the following line:
If not (x > 5 and x 10) then...
How could this be written more simply?
2. Determine whether the following argument is valid:
The le is either a binary le or a text le. If it is a binary le then my
program wont accept it. My program will accept the le. Therefore the le
is a text le.
3. Prove that
a/ (p q) (p q) p
b/ p q p q

1.1.4 Practical projects (Optional)

Rules of inference. Consider an implication of the form

1
,
2
, . . . ,
n

Valid arguments. This implication is understood as

1

2
. . .
n

When all the premises
1
,
2
, . . . ,
n
have true value T, and nd that under
these circumtances also has the value T, then the implication

1

2
. . .
n

is tautology, and we have a valid argument.
1.1. PROPOSITIONAL LOGIC 15
Usages of Rules of inference. Rules of inference
1. enable us to consider only the cases wherein all the premises
1
,
2
, . . . ,
n
are true;
2. used in the development of step-by-step validation of how the conclusion
logically follows from the premises
1
,
2
, . . . ,
n
in the implication of the
form

1
,
2
, . . . ,
n

A few well known valid arguments are summaried next.
I: Rule of Modus Ponens
p (p q) q
Modus Ponens is also called the Rule of Detachment.
Quiz : Can you check p (p q) q is a valid argument?
In the tabular form:
p
p q

q
where the stands for therefore, indicating that q is the conclusion of two
premises p and p q above the line.
Example 5. Consider the following statements.
1/ Barack Obama is a human being.
2/ If x is a human being then x will die.
3/ Barack Obama will die.
II: Rule of Syllogism

(p q) (q r) (p r)
where p, q and r are any statement. In the tabular form, it is written
p q
q r

p r
16 CHAPTER 1. BASIC LOGIC
III: Rule of Modus Tollens

In the tabular form:

p q
q

p
Modus Tollens is a method of denying.
Quiz: Use Modus Tollens to prove the follwing scheme
p r
r s
t s
t u
u

p
for any propositions p, r, s, t, u
IV: Rule of Disjunctive Syllogism

[(p q) p] q
This can be derived from Modus Ponens and the logical equivalence
p q p q
or
p q p q
In the tabular form:
p q
p

q
This rule of inference arises when there are exactly two possibilities to consider,
and we are able to elliminate one of them as being true, i.e. it is false. Then the
other possibility must be true.
1.2. PREDICATE LOGIC 17
V: Rule of Contradiction

Denote F
0
to be a contradiction. You can prove that
(p F
0
) p
is a tautology! This called the Rule of Contradiction, written as:
p F
0

p
The Rule of Contradiction is a key method of establishing the validity of an
argument, the method Proof by Contradiction.
Moreover we can employ this to the case when we wish to prove
(p
1
p
2
. . . p
n
) q
then we can form the validity of the argument
(p
1
p
2
. . . p
n
) q F
0
WHY?
1.2 Predicate logic
1.2.1 What topics will we learn?
A: Predicate logic- concepts of predicate
B: Quantifying a Predicate
C: Quantifying a Predicate- two propositions
D (*): Negating a predicate with many variables and quantiers
A. Propositions versus predicate
Denition 6. A predicate is a statement containing one or more variables; it
cannot be assigned a truth value until the values of the variables are specied.
We will investigate predicate logic later.
Statements containing variables commonly occur in algorithms and computer
programs.

For example, an algorithm might contain the statement x > 5. as the condition
in a control structure such as an If-then.
18 CHAPTER 1. BASIC LOGIC
In this case, however, the truth value of the statement is determined when the
line is executed while the program is being run with a particular set of inputs,
so statements of this type can be treated as propositions.

We now introduce a notation that will allow us to write predicates and

quantiers symbolically. We will use capital letters to denote predicates.
A predicate P containing a variable x can be symbolically written as P(x).
A predicate can contain more than one variable; a predicate P with two
variables, x and y for example, can be written P(x, y).
In general, a predicate with n variables, x
1
, x
2
, . . . , x
n
, can be written
P(x
1
, x
2
, . . . , x
n
).
Predicates are often named with a letter. Furthermore, a functionlike
notation is used to denote a predicate supplied with specic variable
values.
Example 6. n is a perfect square
is a predicate whose truth depends on the value of n.
The predicate is true for n = 4 since 4 is a prefect square [4 = 2
2
],
but false for n = 5 since 5 is not a perfect square [no x Z : 5 = x
2
].
A functionlike notation is used to denote a predicate, e.g. we might name our
earlier predicate P :
P(n) = n is a perfect square

Now P(4) is true, and P(5) is false.

B/ Quantiers- Quantifying a Predicate
There are a couple kinds of assertion one commonly makes about a predicate:
that it is sometimes true and that it is always true. For example, the predicate
Example 7. x
2
0

is always true when x is a real number. On the other

hand, the predicate 5x
2
7 = 0 is only sometimes true; specically, when x =?.
There are several ways to express the notions of
always true and
sometimes true
1.2. PREDICATE LOGIC 19
in English. The tables below give some general formats on the left and specic
examples using those formats on the right.
Always True we use symbol Example
(L1) For all n, P(n) is true. For all x, x
2
0.
n, P(n) is true
or P(n) is true for every n x
2
0 for every x
Sometimes True we use symbol
(L2) There exists an n such that There exists an x
P(n) is true. such that 5x
2
7 = 0
n: P(n) is true
The expressions for all and there exists are called quantiers.
The process of applying a quantier to a variable is called quantifying the
variable.
(L1) For all n, P(n) is true
can be reduced to
n, P(n)
(L2) There exists an n such that P(n) is true
can be reduced to
n, P(n)
Quiz. (1 min)
Write in symbols: There exists an x R such that x < 4.
C/ Quantifying a Predicate- two propositions
For two predicates (i.e. open statements) p(x), q(x) dened for a prescribed
universe U, consider the universally quantied statement
x U [ p(x) q(x) ]
20 CHAPTER 1. BASIC LOGIC
or in short
x[ p(x) q(x) ].
We have
Denition 7. if the implication p(a) q(a) is true for each a in the universe
U then we say that p(x) logically implies q(x), and can write
x[ p(x) q(x) ].
We similarly dene the logically equivalent statement
x U [ p(x) q(x) ].
D. Practicce: Negating a predicate with many variables and
quantiers
We can see that forming the negation corresponds to negating the predicate
and changing the quantier. We can express this observation as a law of
predicate logic:
Rules for negating a predicate.
Rule 1 : [ xP(x) ] x[ P(x) ]

There is a second law, be seen of as the dual of the rst, for negating a
proposition containing there exists. It also corresponds to negating the
predicate and changing the quantier:
Rule 2 : [ xP(x) ] x[ P(x) ]

Practice 1: we want to apply the connective not to the following proposition:

All swans are black.
Applying not to a proposition is called negating the proposition.
Hint: The original proposition can be written in symbols:
x, P(x)
where P(x) is the predicate Swan x is black.
Suggested Solutions:
1.2. PREDICATE LOGIC 21
- Here is one way of forming the negation:
It is not true that all swans are black.
- Or, more simply:
Not all swans are black. There is a swan that is not black.

Practice 2: Write down the negation of the following proposition:

For every number x there is a number y such that y < x.

Summary
Rule 1 : [ xP(x) ] x[ P(x) ]
Rule 2 : [ xP(x) ] x[ P(x) ]
Rule 3 : [ xy P(x, y) ] xy [ P(x, y) ]
Rule 4 : [ x(P(x) Q(x)) ] x[ P(x) Q(x) ]
1.2.2 Problems- Solving Hints
I) Quantiers with many variables
A variable which has been quantied is said to be bound. A variable that
appears in a predicate but is not bound is said to be free.
Mixing Quantiers. Many mathematical statements involve several
quantiers.
Example 8. Write the following two propositions in symbols:
For every number x there is a number y such that y = x + 1.
There is a number y such that, for every number x, y = x + 1.
Can you symbolize the statement in two ways:
Every American has a dream.
Hint: let A be the set of Americans, let D be the set of dreams, and dene the
predicate
H(a, d) to be American a has dream d..
II) Predicate with many variables.
22 CHAPTER 1. BASIC LOGIC
In the specication of a system for booking theatre seats, B(p, s) denotes the
predicate person p has booked seat s. Write the following sentences in
symbolic form:
1. Seat s has been booked.
2. Person p has booked a (that is, at least one) seat.
3. All the seats are booked.
4. No seat is booked by more than one person.
III) If And Only If (I) Proofs
A statement of the form A if and only if (i) B means
A implies B and B implies A or in mathematical symbols
(A B) (B A) = (A B)
So there are actually two proofs to give.
The proofs can also be written as a single proof of the form
A i C i D i i B,
where each i statement is clear from previous information.
Practice 1: x is even if and only if x
2
2x + 1 is odd.
Practice 2: x
2
is odd if and only if x is odd.
IV) Symbolically proof.
Consider the following statements
Nguyen is playing a game.
If Nguyen is playing a game then he is not solving his home works.
If Nguyen is not solving his home works then his father will not buy him a
motorbike.
Can you prove that
Therefore, Nguyens father will not buy him a motorbike.
symbolically?
Chapter 2
Set Theory
2.1 Sets- Ways of describing things
2.1.1 Motivation
How could you write or represent all national soccer teams s taking part world
soccer matches in South Africa in the year 2010?
2.1.2 Concepts
A Set is a collection of some things. Set is a fundamental notion.
For instance,
1/ set A = {shirt, pull}, set B = {jean, trouser, sport}.
set of naturals N = {1, 2, 3, . . .},
We write shirt A, trouser B, but must write trouser N!
2/ For ex., to the example of world soccer matches in South Africa write
S = {s : a property P(s) is fullled, that s satises}
where P(s) = national soccer teams s taking part matches in South Africa.
Notation.
If S is a set and x is a member or element of S we write x S. Othewise
we write x S.
The set with elements x
1
, , x
n
is denoted {x
1
, , x
n
}.
The empty set with no elements is denoted {} or .
A set with one element is called a singleton. e.g., {a} is a singleton.
23
24 CHAPTER 2. SET THEORY
A sequence u, an ordered set made by elements a, b, c..., is written by
u = (a, b, c, . . .), e.g. u = (s
1
, s
2
, . . . , s
80
) is a sequence.
If an ordered set has only two elements it is called a pair.
Example 1: a daily outt (pull, jean) is such a pair.
A sequence is not a set, although they are made from the same elements.
Example 2: Hence, the sequence
(An, Binh, Cuong) = {An, Binh, Cuong}, which is a set.

Operation on Sets.
Forming new set from two sets intersection
The intersection E F, of sets E and F, consists of their common elements:
E F = {x : x E and x F}
Forming new set from two sets Cartesian product
The Cartesian product of two sets A and B, denoted by AB, consists of all
pairs (a, b) where a A and b B.
Set equality and subsets
Subsets. The set A is a subset of B, denoted A B, means every element
of A is an element of B, i.e. [ for all s, if s A then s B ]
The power set of a set S, denoted power(S) or P(S) is the set of all
subsets of S.
Two sets A, B are equal, denoted A = B if they have the same elements.
2.1.3 Examples
Example 0/ Well-known number sets are
the natural numbers {0, 1, } by N, the set of integers is denoted by Z,
the rational numbers by Q, and the real numbers by R.
Q =
_
m
n
: m, n Z, n = 0
_
1/ Could you nd the intersection E F when
E = [1; 4] := {x R : 1 x 4},
2.1. SETS- WAYS OF DESCRIBING THINGS 25
i) F = [2; 6),
ii) F = (0; 10), and
iii) F = [0; )?
2/ List all members (elements) of AB if given A = {shirt, pull},
B = {jean, trouser, sport}
3/ Quiz. Determine the set P(X) P(Y ) if you know
X = {a, b, 1} and Y = {u, a, b},
where P(S) is the set consisting all subsets of S.
2.1.4 Key Mathematical Symbols
SYMBOL MEANING Note
Belong to relation
Not belong to relation
imply; it follows that logic operator
Implied by logic operator
Equivalent to; if and only if logic operator
N Natural numbers set notation
Q Rational numbers set notation
Z Integer numbers set notation
R Real numbers set notation
For every quantier
There exists quantier
26 CHAPTER 2. SET THEORY
2.2 Relations
* A binary relation R between A and B is a rule that associate some
elements a A with certain elements b B.
Equivalently, it is a subset R of the product AB; that means
R = {some (a, b) : a and b satisfy some condition} AB
* Inx Notation: If R is binary, we can use inx to represent pairs in R.
For example, call R the is square of relation on N.
R = {(0, 0), (1, 1), (4, 2), (9, 3), . . . , (n
2
, n), . . .} = {(n
2
, n)|n N}.
In this example, we have (9, 3) R. So we can also write
R(9, 3) or 9R3 or 9 is square of 3.
-
* In general if x
i
A
i
, for all i = 1..n, the set of all n-tuples (x
1
, , x
n
) is
denoted
A = A
1
A
2
A
n
= {(x
1
, , x
n
)}
A n-ary relation R is a subset of A = A
1
A
2
A
n
If R is a relation and (x
1
, , x
n
) R, we write R(x
1
, , x
n
).
n-ary relations Application in CS: Relational Databases. A
relational database is a relation where the indexes of a tuple have associated
names called attributes.
Example. Set n = 3,
A
1
a set of names, A
2
a set of majors, A
3
a set of credits,
and let Students be a ternary relation between the sets A
1
, A
2
, A
3
Students = {(x, y, z) | x is a Name, y is a Major, and z is Credits}.
Students is also called a relational database with 3 elds A
1
of strings, A
2
of
strings, and A
3
of natural numbers.
2.2.1 Equivalence Relation
Consider a binary relation R on A (between A and A), it is just a subset R of
the product AA, i.e.
R = {some pairs (a, b) AA : a A b A}.
R is called an equivalence relation on A if it is reexive, symmetric and
transitive, due to the below meanings.
2.2. RELATIONS 27
1. Reexive property: aRa for all a A.
2. Symmetric property: For all a, b A, if aRb then bRa.
3. Transitive property: For all a, b, c A, if aRb and bRc then aRc.
Theorem 8. An equivalence relation R provides a partition of the set A, that
partition is formed by the set of all equivalence classes of R.
2.2.2 Partial orders
Consider a binary relation R on A (between A and A), it is just a subset R of
the product AA, i.e.
R = {some pairs (a, b) AA : a A b A}.
R is called a partial ordering relation on A if it is reexive, anti-symmetric
and transitive, due to the below meanings.
1. Reexive property: aRa for all a A.
2. Anti-symmetric property: For all a, b A, if aRb and bRa then a = b.
3. Transitive property: For all a, b, c A, if aRb and bRc then aRc.
If we could determine a partial ordering relation R on A then (A, R) is called a
poset.
2.2.3 Example
Relation of weekly approriate outt (for IU students from Monday till Saturday)
R = {(a, b) : a and b make a right outt}, i.e.
R = {Monday, Tuesday, . . . , Friday} =
{(shirt, jean), (shirt, trouser), (pull, jean), (pull, trouser), (pull, sport)}
Is R an equivalence relation? a partial ordering relation R on A? Why?
2.2.4 Problems- Solving Hints
A) Extract infomation from Relational Databases.
Recall that A
1
a set of names, A
2
a set of majors, A
3
a set of credits,
and let Students be a ternary relation between the sets A
1
, A
2
, A
3
Students = {(x, y, z) | x is a Name, y is a Major, and z is Credits}.
1/ Who are the student majoring in CS in senior class (4-th year)?
28 CHAPTER 2. SET THEORY
Set y = CS (constant string), and set z 90 for 4-th year students, then
senior-Students = {(x, CS, z) Students such that z 90}
2/ What is the history department database of names and their credits?
History-Students = {(x, z) | (x, His, z) Students } A
1
A
3
In Relational Databases theory, History-Students is called a projection on
A
1
A
3
with respect to the direction y = His.
B) Poset Diagram (Hasse Diagram): A graph representing a poset but with
only immediate predecessor edges, and the edges are oriented up from x to y
when x < y.
Quiz: can you show the Hasse Diagram of ({2, 3, 4, 8, 12, 15}, |)?
2.2.5 Practical projects (Optional)
Let A be a set of nine team of students, in a Green Summer Actions at
HCMUS, where the performance of team t
i
is denoted by p
i
Q
+
, for i = 1..9.
Suppose that the performances are distinct and sorted in increasing order, that
is p
1
< p
2
< p
3
< . . . < p
8
< p
9
, and the corresponding teams in A are denoted
as A = {t
1
, t
2
, t
3
, . . . , t
8
, t
9
}. We dene a binary relation on A by
t
i
t
j
p
i
p
j
.
Could you determine
a) whether (A, ) is a partial order set? Total order set? Explain why?
b) how many subsets B of A that have precisely 5 elements (teams), consist
the worst team t
1
, and not consist teams t
2
, t
3
?
c) how many subsets C of A having precisely 5 elements and the weakest
team of C is t
4
?
2.3. FUNCTIONS- WAYS OF TRANFORMING THINGS 29
2.3 Functions- Ways of tranforming things
2.3.1 Motivation
A) The idea of function is one of the most fundamental concepts in modern
mathematics. A function expresses the hypothesis of one quantity depending on
(or being determined by) another quantity. For example:
(i) bone mass is dependent on age of subject;
(ii) height is dependent on races etc.
If a function f assigns a value y in the range to a certain x in the domain, then
we write: y = f(x) where
x is called independent variable and y is dependent variable (although this
terminology is sometimes controversial).
B) When HCMCs Environmental Protection Agency (EPA) found a certain
company C (paper mill or cleaning chemical factory) dumping contaminated
liquid into Saigon River,
it ned the company C $12500 plus $100 per day
until the company compiled with national pollution rules.
Request: express the total ne as a function of the number x of days the rm
C continued to violate the national rules. Determine the slope and give your
remark about it.
2.3.2 Theory
Domain and range.
In general, given two sets A (domain) and B (contains range), function f from
set A to set B, is a special rule, such that each value a A gives rise a unique
value f(a) = b. We describe functions by few ways:
1. use an algebraic formula: function f : A B is given by a formula
f(a) = b,
2. by relation notation:
f = {(a, b) : each value a A gives a unique image b B}
or
f = {(a, f(a)) : a A and !f(a) B}
* The domain of f is A, denoted by Dom(f), consists of all a such that f(a) is
computable,
* b = f(a) is the image of a under f. The set of values b is called its range or
image, written Range(f).
30 CHAPTER 2. SET THEORY
Composite function.
Suppose that f, g are real-valued function with domains Dom(f) and Dom(g)
respectively. The function h := f g given by
h(x) := (f g)(x) = f[g(x)]
is called the composite of f with g.
The domain of f g is the set of all x Dom(g) such that g(x) Dom(f).
Note that it can happen that f g = g f. [why? try visualize the
diagram]
2.3.3 Examples
Ex 1: A = {1, 4, 7, 8, 10}, B = {2, 8, 14, 16, 17, 20, 35} N,
the doubling function is dened
1. by algebraic formula, can be written shortly by
f(a) = 2a, a A
2. by relation notation f = {(1, 2), (4, 8), (7, 14), (8, 16), (10, 20)},
Remark that graphing/sketching a function f on 2-dimensional
plane Oxy (2D-plane) means representing f by relation notation.
2.3.4 Problems- Solving Hints
A/ Number Theory- basic concepts
Division algorithm. For a, b Z with b = 0, there are unique pair of
integers q, r Z such that
a = bq +r with 0 r < |b|
Euclids algorithm for nding gcd(x, y) [try it on Maple!]
Example. Find gcd(189, 33)
The mod Function. For a, b Z with b > 0, apply the division
algorithm to get a = bq +r with 0 r < b. The remainder r is the value
of the mod function applied to a and b:
r = a mod b
2.3. FUNCTIONS- WAYS OF TRANFORMING THINGS 31
How to get a formula for r in terms of a and b?
Hint: we solve the equation for q = a/b r/b, employing the fact that
0 r < b
B/ Constructing Function A function f : R R given by f(x) = x
2
, a
function g : R R given by g(x) = 4x + 3
The composite function go f is determined by the formula ??
The composite function fo g is determined by the formula ??
C/ Specic properties of functions Let f : A B be a function.
Injective (also called one-to-one): Distinct elements in A map to distinct
elements in B; formalizing as
a = b f(a) = f(b)
Is f : N
8
N, be dened by f(x) = 2x mod 8
an injective function?
Surjective (also called onto): The range is the codomain. That is each
b B has the form b = f(a) for some a A.
Bijective (also called one-to-one and onto): Both injective and surjective.
That is
any b B has the form b = f(a) for a unique a A.
Prove: the function f : (0, 1) (2, 5) dened by f(x) = 3x + 2 is a
bijection.
2.3.5 Practical projects (Optional)
1. Piecewise functions are functions of the form
y =
_
f
1
(x) when x I
1
, (a real interval)
f
2
(x) when x I
2
, (another interval)
Sketching the graph of the piecewise function
y =
_
5
2
x
1
2
if 1 x 1
x
2
+ 4x + 5 if x > 1
2. Basic graphs and how to nd their intersection.
Given two graphs G : y = f(x) . . . () and H : y = g(x) . . . (), their
intersection G H can be determined by solving the system of equations
32 CHAPTER 2. SET THEORY
f(x) = g(x) . . .
to nd solutions x
i
.
The intersection G H = {M(x
i
, y
i
) : M G and M H}, that is
G H = {M(x
i
, y
i
) : f(x
i
) = g(x
i
) and y
i
= f(x
i
)}
Application of intersecting graphs: Break-even points: A cable
television company estimates that with t thousand subscribers, its
monthly revenue and cost (in thousands of dollars) are
R(t) = t
2
+ 3t 2 . . . (G) and
C(t), be given by 4t + 8s 6 = 0 . . . (H).
Find the intersection of the curve G and the line H on the Ots-plane?
Notation: The companys break-even points are the number of
subscribers t
0
at which R(t
0
) = C(t
0
).
3. Inverses: If f : A B, b = f(a) is a bijection, then there is an inverse
function g : B A dened by (an inverse action)
g(b) = a i f(a) = b
The inverse of f is denoted by f
1
.
E.g., the function f : (0, 1) (2, 5) dened by f(x) = 3x + 2 =: y is a
bijection, so the inverse of f is denoted by f
1
= g : (2, 5) (0, 1) dened
by ??
2.4 Integers and Algorithms on Integers
2.4.1 Review- Motivation
We start with the basic concept of a number.
The Natural Numbers, denoted as N, are the numbers 1,2,3,... These
are closed under addition (e.g., 3 + 29 = 32 N) .
Introducing the notion of subtraction makes us enlarge N to get the
Integers, denoted as Z, positive or negative or 0, so that our numbers are
closed under the operation of substraction.
2.4. INTEGERS AND ALGORITHMS ON INTEGERS 33
Z is closed under multiplication as well; which is to say that the product
of two integers is an integer (e.g., 3.(9) = 27 Z) .
To obtain a set of numbers closed under division:
we must enlarge Z to get the Rational Numbers Q, which are fractions
of the form a/b where a is in Z and b is in N.
We then have the extension diagram
N Z Q
The numbers between 0 and 1 can each be represented as a decimal point
followed by an innite string of digits, each digit being one of 1, 2, 3, ..., 9, 0.
Rational numbers in Q repeat themselves endlessly after some point: for
example 1 / 4 is .250*
(where the star means that you repeat the starred digit endlessly)
1 / 3 is .3*, 57 / 100 = .570*.
1 / 7 = .(142857)*,
Hence, we have the extension diagram
N Z Q R
Irrational numbers (not in Q, but still in R) do not do so, for example
e = 1.27... the Nepe base number of logarithm
= 1.414... = 2.90
o
the unit of measuring angle
2.4.2 Introduction to Algorithms
Learn chapter 14, key text by BIGGS, on Algorithms and Eciency, pages 159
171.
2.4.3 Basic Integer Theory
Divisibility algorithms
Counting systems- Arithmetic operations
Congruences linear congruences, the additive group Z
n
Greatest common divisors The Euclidean algorithm
Decompsing an integer by relatively prime numbers
34 CHAPTER 2. SET THEORY
Divisibility algorithms
Minimum principle: In a total ordering set (S, ), for any subset = A S
and A is bounded below, then there exists a smallest element of A, that is:
b S, a A : b a = a
0
A : a
0
a.
Divisibility: for a, d Z, we say d is a divisor of a i:
q Z : a = qd
Division algorithm: Let a, d Z with d 1. Then
!(q, r) Z
2
: a = qd +r where 0 r < d.
q is called the quotient and
r is called the remainder in the division of a and d.
Counting systems- m-adic representation
Fix an m Z and m 2, any n Z can be represented uniquely in the form:
n = [a
0
a
1
a
k
], means that
n = a
0
+a
1
m+a
2
m
2
+. . . +a
k
m
k
=
k

i=0
a
i
m
i
, where k N
such that
m
k
n < m
k+1
,
1 a
k
m1, and
a
i
N, 0 a
i
m1, for i = 1..k 1.
This is called the m-adic representation of n, a
i
the digits of n to base m.
For example, m = 2, our counting system is said to be binary,
n = 14 = 0 + 1.2
1
+ 1.2
2
+ 1.2
3
then k = 3 and the 2-adic representation of 15 is
[0111].
Group. Let = G, and G
2
= GG.
A binary operation : G
2
G is a map, we write x y := (x, y)
(G, ) is called a group if satises three axioms:
- associativity,
- there exists identity element and
- there exist inverses in G with respect to .
2.4. INTEGERS AND ALGORITHMS ON INTEGERS 35
Moreover, (G, ) is called abelian or commutative if x y = y x.
Ex1: (G, ) = (Z, +) is an abelian group, additively writen..
Ex2: (Q

, ), (R

, ), (C

, ) also abelian, but multiplicative ones.

Subgroup. H G is a subgroup of (G, ) if H also is a group under the same
binary operation as G. We write H G.
Theorem 9 (subgroup of (Z, +) is cyclic). Let H (Z, +), then
!d Z : H = d := dZ.
Greatest common divisor- The Euclidean algorithm
Theorem 10. Let = A (Z, +), then
1. A has a unique greatest common divisor, denoted gcd(A)
2. a
1
, a
2
, . . . , a
k
A and x
1
, x
2
, . . . , x
k
Z such that:
gcd(A) = a
1
x
1
+a
2
x
2
+. . . +a
k
x
k
.
Proof. Put H := {a
1
x
1
+a
2
x
2
+. . . +a
k
x
k
: a
i
A and x
i
Z}.
Greatest common divisor- specic cases.
If = A = {a
1
, a
2
, . . . , a
k
} Z is nite, then we write
gcd(A) = (a
1
, a
2
, . . . , a
k
).
For instance, A = {12, 15} then gcd(A) = (12, 15) = 3.
Integers a
1
, a
2
, . . . , a
k
are called relatively prime i (a
1
, a
2
, . . . , a
k
) = 1,
and pairwise relatively prime i
(a
i
, a
j
) = 1 for i = j.
For instance, 2, 4, 5 are called relatively prime since gcd(A) = (2, 4, 5) = 1.
But 2, 5, 7 are called pairwise relatively prime since (2, 5) = (2, 7) = (5, 7) = 1.
The Euclidean algorithm computes gcd.
Given a, b Z and b 1.
The Euclidean algorithm aims at constructing a sequence r
i
, where
(a, b) = (r
0
, r
1
) = (r
1
, r
2
) = . . . = (r
n1
, r
n
) = (r
n
, r
n+1
) = r
n
and r
i1
= r
i
q
i1
+r
i+1
, 0 r
i+1
< r
i
, 1 i n.
For e.g. try (30, 12) = (12, 6) = (6, 0) = 6.
Ex 3: let apply the Euclidean algorithm to nd (574, 252).
36 CHAPTER 2. SET THEORY
Congruences linear congruences- Chinese remainder theorem
Introduce the concept of ring (R, +, ).
Introduce Z
m
:= Z/mZ
Fact: (Z
m
, +, ) is a commutative ring for every integer m 2.
The Chinese remainder theorem is very popular in the sense that it is found in
many applications. In particular, we can use it
1. when factorizing polynomials over nite elds and over the integers Z,
employed in Cryptography;
2. in computing balanced factorial designs in Industrial Statistics and
Manufacturing,
3. in many other applications that require a decomposition of a big ring R in
terms of its components ...
Application 1: Public Key Cryptography: See page 77 of Nathanson for
an introduction. Read RSA cryptosystem. See next section for background.
2.4.4 Application of Integers in Cryptology
Write N
n
:= {1, 2, 3, , n}. As a result N
26
:= {1, 2, 3, , 26}
Ciphers and the mod function. Let the letters from a to z be represented by
0, 1, ..., 25, respectively. Any bijection f : N
26
N
26
can act as a cipher and its
inverse f
1
(also a bijection) can act as a decipher.
Well-known ciphers: additive, multiplicative and ane.
Example (additive cipher).
f(x) = (x + 2) mod 26 and f
1
(x) = (x + 24) mod 26.
Example (ane cipher).
f(x) = (5x + 1) mod 26 and f
1
(x) = (5x + 5) mod 26.
Hash Functions setting. Why we know this? Due to
Theorem 11 (Mod and Inverses). Let n > 1 and f : N
n
N
n
be dened by
f(x) = (ax +b) mod n.
Then
2.4. INTEGERS AND ALGORITHMS ON INTEGERS 37
f is bijective i gcd(a, n) = 1. If so, then
f
1
(x) = (kx +c) mod n where f(c) = 0 and 1 = ak +nm.
Some ciphers keep one or more letters xed. For example, the multiplicative
cipher f(x) = 3x mod 26 has f(0) = 0, so it sends the letter a to itself. Letter
a is named xed letter or xed point of the cipher f(x) = 3x mod 26.
The below theorem can be used to construct ciphers with no xed letters.
Mod and Fixed Points. Let n > 0 and f : N
n
N
n
be dened by
f(x) = (ax +b) mod n.
Then f has no xed points i gcd(a 1, n) does not divide b.
In other words, f has xed points i gcd(a 1, n) divides b.
Hash Functions
A hash function maps a set S of keys into a nite set N
n
of table indexes:
h : S N
n
The table is called a hash table.
Collisions occur if the function is not injective.
If there are no collisions, then any key in S is mapped to the index where the
information is stored without any searching.
Example. Let S be the students in a class and let h : S N
366
be dened by
letting h(x) be the birthday of x. If two people have the same birthday, then a
collision occurs.
Resolving Collisions by Linear Probing
If a collision occurs at index k, then some key is placed in location k and the
other colliding keys must be located elsewhere.
Linear probing is a technique to search/probe for an open place in the table
by looking linearly at the following places, where g is a xed gap:
(k +g) mod n, (k + 2g) mod n, , (k + (n 1)g) mod n.
38 CHAPTER 2. SET THEORY
2.4.5 Examples
Example 1. Let S = {jan, feb, mar, apr, may, jun} and let h : S N
6
be
dened by h(xyz) = p(x) mod 6, where p(x) is the position of x in the alphabet
(p(a) = 1, , p(z) = 26).
Place the keys from S into a hash table by rst placing jan, then feb, and so on.
h(jan) = p(j) mod 6 = 10 mod 6 = 4.
So place jan in position 4 of the table. Continue the process to get
h(mar) = 1, h(apr) = 1 (collision with mar).
Find the table showing the result of resolving collisions by linear probing with a
gap of 1.
2.4.6 Problems- Solving Hints
Write N
n
:= {1, 2, 3, , n}
1. Is f : N
5
N
5
be dened by f(x) = (4x + 1) mod 5 a bijection? If yes,
nd its inverse.
2. The additive cipher f(x) = (x + 2) mod 26 and the ane cipher
f(x) = (5x + 1) mod 26 both have or dont have xed points?
2.4.7 Practical projects (Optional)
For a group of max 4 students: Conduct a small research on RSA encryption-
How to keep your information on computer nets from eardropers?
Chapter 3
Counting Techniques.
3.1 Basic counting
We can compare the cardinalities of two sets by using
- functions as injection, surjection and bijection;
- other set-based methods as inclusion- exclusion or Union Rule;
- Pigeonhole rule and others ...
3.1.1 Motivation
Practical questions that you could face in your daily life are:
1. Suppose that you wish to take two fruits for your lunch when go to school.
Known that you have three bananas, four apples and two mangos, what is
the number of ways that you can select two pieces of fruit of dierent
types?
2. Ten soccer teams are competing in a V-league. In how many dierent
ways can the rst three places be lled?
3. A customer surveyor at your rm asked respondents to indicate their
highest level of education. The only three choices in the survey are high
school, college, and other. If 31% indicated high school and 49% indicated
college, then how much the percentage of respondents who chose the other
category is?
4. A security key word w of length 10 is formed by two sub-strings w
A
and
w
B
of length 3 and 7, that is w = w
A
w
B
(concatenating two sub-strings).
Known that repetition is allowed, w
A
is made by distinct characters of the
set S = {a, b, c, d, e, f} and w
B
is made by characters of the set
R = {1, 2, 3, u, v, x, y, z}
How many security key words w can be created totally?
39
40 CHAPTER 3. COUNTING TECHNIQUES.
5. * Suppose you cut a chocolate bar of rectangular shape with the length m
and the width n by a horizontal cut (along the length) or vertical cut
(along the width) into small square pieces of size x.
Known that only one cut is allowed in each cutting step, and that m, n, x
all are eligible positive integers.
In your opinion, what is the correct relationship between p, x, m, n?
If the length m = 81, the width n = 10, and the size x of square chocolate
pieces satises that 1 < x < 5, then what is the number of small chocolate
pieces p you can make?
3.1.2 Theory on Countability
Most used concept is:
|A| = |B| or A B means there is a bijection between A and B.
(a0) A set S is nite (also called countably nite) if there exists an n N so
that |S| = |N
n
|
(a1) Say that S is countably innite when |S| = |N|
Denition 12. We say
(a) a set S is countable if |S| = |N
n
| or |S| = |N|
(b) a set S is uncountable if (a) does not happen!
Obviously,
1. S is countable if |S| |N|; S is uncountable if |N| < |S|
2. S N implies S is countable, [the inverse is not correct, e.g. |Z| = |N|, say Z
is countable but Z N]
Cantors Result: |A| < |power(A)| for any set A.
Proof : use Diagonalization Technique.
Example. power(N) is uncountable because |N| < |power(N)|.
Inclusion-exclusion, counting practice.
Recall that we have a few basic rules
Product Rule: |AB| = |A||B| and |A
n
| = |A|
n
.
Inclusion-Exclusion or Union Rule: |A B| = |A| +|B| |A B|
Dierence Rule: |A\ B| = |A| |B|
3.2. PROBLEMS- SOLVING HINTS 41
Pigeonhole and division rules.
Pigeon Hole Principle. If m things are put into n places and m > n, then
one place has two or more things (a collision occurs).
Another way to say this is that:
if A and B are nite sets with |A| > |B|,
then there are no injections from A to B.
Example. In HCM City there are two people with the same number of hairs on
their heads. Everyone has less than 8 million hairs on their head and the
population of HCM City is more than 8 million. So the pigeon hole principle
applies.
Binomial theorem, combinatorial identities
See text books.
3.1.3 Examples
Example. Consider a map f : Z N being determined by:
f(x) =
_
2x 1 when x I
1
= (0, +)
2x when x I
2
= (, 0]
* Prove f is a bijection, nd f
1
.
* Can we conclude that N <
c
Z? What else?
3.2 Problems- Solving Hints
3.2.1 Basic problems
from text books.
P1:
A permutation f on a nite set S of n elements is a bijective (one-to-one)
mapping f from S to itself. Find the total number of permutations on S
with an elucidated reasoning.
(Division rule). If A, B both are nite sets and f : A B is a mapping.
Check the equality
A =

bB
f
1
(b).
Does this represent a partition of the set A? Elucidate your reasoning.
Evaluate |A| if you know that |f
1
(b)| = k N

for all b B.
42 CHAPTER 3. COUNTING TECHNIQUES.
3.2.2 Practical projects (Optional)
Algorithmic problem solving: First Tool: Invariants
Invariant means a not-changing property of a process or system when that
process evolves. Similar terms of invariant are law and pattern. In
mathematical modeling and algorithmic problem solving, very often
* you wants to model a system in a concise way as much as possible (of course
after droping unimportant aspects to make the model tractable);
** then your goal is to nd some solution or conclusion from making logical
reasoning on the model by mathematical techniques or by simulation it.
The crucial structures/objects/property that you wish to nd out and employ
during both two phases above are xed pattern or invariant during the whole
process evolution.
Reason: these xed rules could help you to keep the model within tractable
range, and moreover to restrict the search domain of solutions.
Example 9. Cutting a chocolate bar of a rectangular shape by a horizontal cut
or vertical cut into unit-size squares provides an invariant between the number
of cuts c and the number of pieces p:
? =??
no matter how large the bar is!
Denition 13 (System invariants). Invariants of the system S are specic
constraints or properties that do not change when the system factors (variables)
change their values [to ensure S stability, existence ...] during the evolution of
concerned system.

How to nd invariants? But how to gure out interesting invariants?

This activity depends very much on the type and complication of your system
or process. You have to detect and employ any meaningful relationship between
the key components/factors/variables that make the process running or working.
The more various models and application domains you face, the more
mathematical methods that you have in your hands, the better the
invariants could be found and exploited.
Fact 1.
a) Try to represent invariants mathematically, not by words!
b) When you think that some rules could be invariants, you have to prove your
thoughts by logical reasoning.
3.2. PROBLEMS- SOLVING HINTS 43
Project A: (Application 5 from Motivation). Known that only one cut is
allowed in each cutting step, and that m, n, x all are eligible positive integers.
In your opinion, what is the correct relationship between p, x, m, n?
If the length m = 81, the width n = 10, and the size x of square chocolate pieces
satises that 1 < x < 5, then what is the number of small chocolate pieces p you
can make?
44 CHAPTER 3. COUNTING TECHNIQUES.
Chapter 4
Induction and Recursion
We discuss specic usages of Sets and Relations in Computing and
Computational Mathematics.
Recursively Dened Functions
Total or Linear Orders
Inductive Proof: The Basis and Principle of Mathematical Induction
4.1 Inductively Dened Sets
4.1.1 How to do?
To dene a set S inductively is to do three things:
1. Basis: Specify one or more elements of S.
2. Induction: Specify one or more rules to construct elements of S from
existing elements of S.
3. Closure: Specify that no other elements are in S (always assumed).
45
46 CHAPTER 4. INDUCTION AND RECURSION
Note: The basis elements and the induction rules are called constructors.
4.1.2 Examples
Example 1. Find an inductive denition for S = {3, 16, 29, 42, }.
Solution: Basis: 3 S.
Induction: If x S then x + 13 S.
The constructors are 3 and the operation of adding 13.
Example 2. Try at home, nd an inductive denition for
S = {3, 4, 5, 8, 9, 12, 16, 17, 20, 24, 33, }.
Example 3. Describe the set S dened inductively as follows:
Basis: 2 S;
Induction: If x S then x 3 S
Solution: DIY (Do It Yourselves)
Example 4. Describe the set S dened by:
Basis: a, b S;
Induction: x S implies f(x) S
Solution: S = {a, (a), ((a)), } {b, (b), ((b)), } =?
Quiz: Is N an inductively dened set?
4.2 Recursively Dened Functions and Procedures
A function f is recursively dened if at least one value f(x) is dened in terms
of another value f(y), where x = y.
Similarly, a procedure P is recursively dened if the action of P(x) is dened
in terms of another action P(y), where x = y.
4.3. INDUCTIVE PROOF 47
Techniques
Technique for recursive denitions when the argument domain is inductively
dened.
1. Specify a value f(x), or P(x), for each basis element x of S.
2. Specify rules that, for each inductively dened element x S, dene the value
f(x), or action P(x) in terms of previously dened values of f, or actions of P.
Notice that N is an inductively dened set, due to the fact:
0 N, and n N implies n + 1 N.

Example 5. Find a recursive denition for the function f : N N dened by

f(n) = 0 + 3 + 6 + + 3n.
Solution: we need to give f(0) a value in N and we need to dene f(n + 1) in
terms of f(n).
The given denition of f tells us to set f(0) = 0.
To discover a denition for f(n + 1) we can write
f(n + 1) = (0 + 3 + 6 + + 3n) + 3(n + 1) = f(n) + 3(n + 1).
So we have a recursive denition for f:
f(0) = 0; f(n + 1) = f(n) + 3(n + 1).
4.3 Inductive Proof
What do we believe about nonempty subsets A of N?
48 CHAPTER 4. INDUCTION AND RECURSION
Since (N, <) is well-ordered, and in fact it is linear (means totally ordered
set), it follows that every nonempty subset A has a least element
Principle of Mathematical Induction
The Basis for Principle of Mathematical Induction (PMI) is based on:
Let A N, 0
A
A, and suppose also that k A implies k + 1 A.
Then A = N.
In practice, we associate A N with a predicate P(n).

Principle of Mathematical Induction (PMI).

Let P(n) denote a statement for each n N. P(n) denotes a statement for each
n N. To prove P(n) is true for each n N do the following steps.
1. Show P(0) is true.
2. Show that if P(k) is true then P(k + 1) is true.
Remark. PMI also works for statements P(n) where n {m, m+1, }. In this
case, the least element is m, so step 1 is modied to show that P(m) is true.
4.4 Problems- Solving Hints
Students should try to solve at least 80% of the following items.
1. For each mapping f : A B below, provide your justication that
whether it is an injection, subjection or bijection? When it is a bijection,
can you nd its inverse:
a) A = B = R, f(x) = x 29?
4.4. PROBLEMS- SOLVING HINTS 49
b) A = [4, 9], B = [21, 96] and f(x) = x
2
+ 2x 3?
c) A = B = N, f(x) = x(x + 1)?
2. (Computing Lab) Provide an algorithm and, if possible, write a computer
program listing all subsets of ve elements of the set S = {1, 2, 3, . . . , 40}.
3. Let n N

, and U
n
= {a N : a|n(a is a divisor of n)}. On U
n
we dene
the relation:
x < y x|y.
* Prove that: (U
n
, <) is a poset (partial ordered set with respect to <)
* Draw the Hasse diagram of (U
15
, <) and of (U
16
, <)
4. Draw the Hasse diagram of poset (P(X), ) [the power set P(X) with
ordering ] when X = {a, b, c} and when X = {f, g, c, w}
5. Given a poset (A, ), A is nite, and let x, y be two arbitrary elements of
A. Recall that
The Hasse diagram of (A, ) include:
a set P of points in 2D-plane which correspond one-one with A, i.e.
there is a bijection between A and P
a set Q of directed arcs connecting some pair of distinct points:
x = y are connected by an arc from x to y if:
x < y and there is no z A : x < z < y
50 CHAPTER 4. INDUCTION AND RECURSION
Topics for Presentation
RSA encryption How to keep your information on nets from eardrops?
Integer Programming How to nd solution whose components are integers?
Planar graphs How to design computer board rightly?
Graph coloring How to design or draw maps mathematically correct?
Bipartite matching How to assign jobs to suitable workers?
Max-ow Min-cut algorithm by Ford-Fulkerson How to push most uid
subtances (as oil, trac, ...) on a directed net?
DOE: How to compute fractional designs
Shortest paths
Chapter 5
Graph Theory and Models
Better tools for representing discrete strutures. What interesting
structures else?
Graphs- an introduction
Undirected graph
Directed Graphs
Trees
5.1 Undirected Graphs
5.1.1 Basic concepts and basic operations
Basic concepts.
A undirected graph G = (V, E) consists of
a vertex set V = {1, 2, 3, , i, , n} of nodes or vertices; and
a set E of pairs of elements in V .
We call Es members the edges of G.
51
52 CHAPTER 5. GRAPH THEORY AND MODELS
Adjacency matrix A(G) = [a
ij
] is a 0-1 (Boolean) matrix, where
a
ij
= 1 when (i, j) E
and
a
ij
= 0 when (i, j) E.
Example. Let V = {a, b, c}, E = {(a, b), (b, a), (b, c)}, then G = (V, E) is a
graph, called complete graph K
3
on 3 vertices.
Quiz. Describe complete graphs K
4
, K
5
.
Denition 14. In a undirected graph G = (V, E),
degree of a vertex u V , written d(u) is the number of adjacent vertices
v of u.
a simple path P is a path consisting no duplicated edges,
an elementary path is a path that consists no duplicated vertices;
G = (V, E) is connected if for all pair of nodes i, j V , there exists at least a
path connecting them.
G = (V, E) is disconnected if there exists a pair of nodes a, b V for which
no path connecting them.
Undirected Graphs basic operations.
Removing a vertex u implies deleting every edge that has connected with
u;
5.1. UNDIRECTED GRAPHS 53
* a vertex v is called a cut vertex i H := G\ v is disconnected;
Removing an edge e = (u, v) still keeps u, v in G!
* an edge e = (u, v) is called a cut edge or bridge i G\ e is
disconnected.
Undirected Graphs basic properties.
Lemma 15. For an arbitrary undirected graph G = (V, E), the equality
2|E| =

uV
d(u) is valid.

Proof.
Each edge e = (u, v) E is determined by two end vertices u and v. When we
count and sum up the number of edges of the set E, each of them contribute
two to the sum of degrees, therefore the sum must be twice the number of
edges. We obtain that

vV
d(v) = 2 |E|
for any arbitrary undirected graph G = (V, E), including cases of multiple edges
and loops.

5.1.2 Simple Graphs and Connectivity

Simple graphs G = (V, E) are graphs without loop and paralel edges.
Lemma 16. Suppose |V | = n > 0, then
0 d(v
i
) n 1 for all i = 1..n when G = (V, E) is simple.
54 CHAPTER 5. GRAPH THEORY AND MODELS
(G)
1
6
2
3
Figure 5.1: An example of graph G

Connectivity. Recall that a undirected graph G = (V, E) is connected if

between any pair of nodes i, j V we can always nd out a path connecting i, j.
In other words, G = (V, E) is connected if between any pair of nodes i, j V
we can always nd out a path connecting i, j.
Hence, G = (V, E) is disconnected if there exists a pair of nodes i, j V for
which no path connecting them.
* In that case, G = (V, E) is formed by p connected components (viewed as
subgraphs of G), denoted G
1
, G
2
, . . . , G
p
.
* Notation:
G = G
1
G
2
. . . G
p
.
5.1. UNDIRECTED GRAPHS 55
Question: is there any relationship between the number of vertices n, the
number of edges m and the number of connected components p?
A fundamental algebraic result for Simple Graphs.
Theorem 17. Suppose that G = (V, E) be a simple and undirected graph with n
vertices, m edges, and p connected components. We have two constraints on the
number of edges, as follows:
n p m
(n p)(n p + 1)
2
...(a)
Proof. Prove the left inequality by induction on the number of edges m:
m n p p n m
This inequality can be read as: the number connected components p is at least
n m in a simple graph with n vertices, m edges.
m n p p n m = P(m)
P(m) here is a predicate (proposition that depends on variable) with respect to
natural m.
For intance, a graph with n vertices, 0 edge has at least n 0 = n = p
connected components; with 100 vertices, m = 20 edges has at least
100 20 = 80 = p ones.
Formally, let
P(m) = a simple graph with n vertices, m edges must have the number
connected components p is at least n m
56 CHAPTER 5. GRAPH THEORY AND MODELS
be the predicate in m N.
Base step. When m = 0, P(0) holds, since an n-vertex simple graph with
0 edges obviously does have at least p = n n 0 connected components.
Induction step. [P(m) P(m+ 1)] is true?
- Assume that an n-vertex simple graph with m edges has at least n m
connected components; we prove that any n-vertex simple graph H with m+ 1
edges has at least
p
H
= n (m+ 1) = n m1
connected components
- Pick an edge e = (u, v) in H, and delete it from H to obtain a new graph G
having the same vertices, but less than one edge to H, that is G has m edges.
By induction the new graph G has at least p
G
= n m connected components.
- In G, two cases occur:
i) either both u, v belong to one connected component C:
p
H
= p
G
= n m n m1;
ii) or u, v are in distinct connected components, say C
u
, C
v
repectively:
p
H
= p
G
1 = n m1, since C
u
C
v
{e} forms a unique connected
component in H. Therefore the predicate P(m+ 1) is correct.
Conclusion. In a simple graph with n vertices, m edges, the number connected
components p must be at least n m for all m N:
p n m m n p
5.1. UNDIRECTED GRAPHS 57
5.1.3 Bipartite Graph
A graph G = (V, E) is called bipartite graph if:
1. the vertex set V can be split into two disjoint subsets A and B, that is
V = A B, A B = [ written V = AB], and
2. any edge e = (u, v) E must have its end nodes belong in the two subsets
A and B, that is:
e = (u, v) E (u A v B) or (u B v A).
Moreover, for a bipartite graph G = (V, E), if |A| = a, |B = b and every node of
A connects to every node of B, then G is called a complete bipartite graph,
written by G = K
a,b
.
Matching in Bipartite Graph.
Given a bipartite graph G = (V, E) = (AB, E).
A matching M = (W, F) in G is a subgraph of G such that no pair of edges of
M have the same end nodes. In other words,
1/W V, F E, and 2/ for all pair of edges
e = (u, v), f = (x, y) F then (u = x, y) (v = x, y).
A maximal matching M in G is a matching that has maximal number of edges.
Example 10. Assignment problem [jobs to workers, girls to boys, ...] can be
formulated as a problem of nding maximal matchings.
58 CHAPTER 5. GRAPH THEORY AND MODELS
5.1.4 Graph Isomorphism
Two graphs G = (V, E) and H = (W, F) are isomorphic when
1. there exists a bijection f from the vertex set V to the vertex set W, and
2. such that f preserves the adjacency relation, that means:
t = (u, v) E s = (f(u), f(v)) F
The bijection f satisfying Condition 2 is called a graph isomorphism.
Testing whether two graphs are isomorphic is very dicult,
since for graphs having n vertices, there are n! bijection mappings need to be
checked the Condition 2.
Graph Isomorphism- Invariants. Graph invariant is a property P(.) of
graphs that does not change under graph isomorphisms.
Testing whether two graphs are isomorphic, in general is very dicult; but if
you could nd out a graph invariant that does not the same for two concerned
graphs, then you are condent to say the two graphs are not isomorphic!
Why? Just a matter of logic reasoning:
G, H are isomorphic P(G) = P(H) for all invariants P. Hence
( an invariant P : P(G) = P(H)) G, H are not isomorphic.
Graph Invariants include:
5.2. TREE 59
The number of vertices
The number of edges
The number of vertices of xed degree d
The pattern of vertex degrees (after sorting)
The number of simple cycles of xed length l in G and H ...
5.2 Tree
Key concepts are the followings.
A undirected graph G = (V, E) is called a tree if and only if it is
connected and has exactly |E| = |V | 1 edges.
Rooted trees T = (V, E) are trees for which we choose a specic node r
and call it a root of T.
In a rooted tree T with root r, the path length l(r, x) from r to any node
x is the number of edges of that path.
Proposition 18 (Special Features of Tree). Any tree T = (V, E) satsies:
a. there is a unique path between every pair of vertices
b. adding any edge creates a cycle
c. removing any adge disconnects the graph
Equivalent denitions of Tree.
Given a graph G = (V, E) with n vertices and m edges. Then
1. G is tree [with respect to Denition 1/ above]
60 CHAPTER 5. GRAPH THEORY AND MODELS
2. There is exactly one path between any two vertices of G
3. G is connected and removing any adge disconnects the graph
4. G has no cycle (acyclic) and adding any edge creates a cycle
Hint for proving: Conduct the series of proofs:
1. 2. 3. 4. 1.
5.2.1 Binary Trees
In a rooted tree T = (V, E) with root r, we can dene the concepts of parent
and children nodes using the path length l(r, x) from r to a node x.
u is parent of v i uv E and l(r, v) = (r, u) + 1
A node is called a leave i it has no child, otherwise we call an internal
node.
If any internal node has at most two children nodes,
the tree is called Binary Trees.
If any internal node has at most three children nodes,
the tree is called Ternary Trees.
Quiz: can you visualize a tree with 3,4,5 and 6 nodes? Can you visualize a
binary tree with 3,4,5 and 6 nodes?
5.2. TREE 61
5.2.2 Spanning Trees
Any connected graph G contains subgraphs which are trees.
A subgraph of G that is a tree and that includes all the vertices of G is
called a spanning tree.
So any spanning tree has precisely n 1 edges, if G has n verices.
Weighted graphs. We can assign numbers/weights (usually positive numbers)
to all edges of a graph G. Then G is called a weighted graph.
For e.g., G represents a railway network, its weights are costs to build up the
network edges.
Minimum Spanning Trees.
Minimum connector problem a railway network G connects a number of
towns, for which the cost of building a direct link between any two towns
is known.
We wish to determine a subnetwork T of G
covering all towns and has minimal total cost.
Minimum Spanning Tree of G is such an optimal subnetwork!
How to? Use the Kruskal algorithm Idea:
a/ start from an edge with smallest weight,
b/ enlarge the current subnetwork by adding a new edge with currently
smallest weight
c/ check acyclicity of the current subnetwork each step until it has n 1
edges.
62 CHAPTER 5. GRAPH THEORY AND MODELS
5.3 Digraphs
A directed graph (digraph) G = (V, A) consists of vertex set V and a set A of
ordered pairs of elements in V . We call As members the arcs or directed
edges of G. Formally, we understand that for two arcs a, b A that are dened
on two vertices u, v, then we can write
a =

(u, v) = uv, b =

(v, u) = vu A then a = b.
u is the initial (emerging) vertex of a, and
v is the terminal (ending) vertex of a.
Adjacency matrix A(G) = [a
ij
] is a {0, 1} matrix, where
a
ij
= 1 when ij A; and
a
ij
= 0 for other cases.
5.3.1 Path Problem in Directed Graphs
* A path P of length k in a digraph G = (V, A) is a sequence of vertices
P = v
0
, v
1
, . . . , v
k
where v
i1
v
i
A, i = 1..k.
* A cycle C of length k in G = (V, A) is a path
C = v
0
, v
1
, . . . , v
k
where v
i1
v
i
A, i = 1..k and such that v
0
= v
k
* If there is no cycles in G, then G is called acyclic digraph (or PERT chart).
PERT chart is employed intensively in task-scheduling problems, where we want
5.3. DIGRAPHS 63
to model ordered lists or structures that have prerequisite property.
Path Problem (Is there a path from node i to node j?)
Ex., let R = {(1, 2), (2, 3), (3, 4)} on vertices V = {a, b, c}. We can represent R
as an adjacency matrix M where
M
i j
= 1 if i Rj, and M
ij
= 0 otherwise.
If we want an answer to our question, it would be nice to have the adjacency
matrix M

= [a
ij
] for the transitive closure t(R). Then we could simply check
whether a
ij
= 1.
Warshalls algorithm: to compute the adjacency matrix M

for the transitive

closure t(R).
5.3.2 Directed Graphs- The reachability matrix M

* Let M = [m
ij
] be the adjacency matrix of a digraph G = (V, A) with n
vertices. An entry m
ij
= True in M indicates the existence of an arc in G; an
arc is a path of length 1.
The logical Boolean product of M with itself is M
2
= M M, and
a True entry in this matrix M
2
indicates
the existence of a path of length 2.
* In general the matrix
M
k
= M M M, (k times of logical Boolean product)
records all paths of length k in G.
64 CHAPTER 5. GRAPH THEORY AND MODELS
* Dene the logical operator or of two arbitrary Boolean matrices B = [b
ij
],
C = [c
ij
] of the same size to be
Bor C = [b
ij
c
ij
]
forming by the logical of corresponding entries.
We now can dene matrices of the form (B D) or (C K) ...
by combining the two logical operators or and Boolean product
Denition 19. The reachability matrix or the transitive closure matrix M

of
t(R) of the adjacency relation A in G is determined by
M

= M or M
2
or . . . or M
n
5.4. PROBLEMS- SOLVING HINTS 65
5.4 Problems- Solving Hints
Students should try to solve at least 70% of the following items.
5.4.1 Problem 1: Basic concepts and operations
1. Prove that for an arbitrary undirected graph G = (V, E), the equality
2|E| =

vV
d(v) is valid.
2. Prove that in any (undirected) graph without multiple edges and loops
which has at least two vertices, there exist two vertices with identical
degrees.
3. Provide the denitions of: i) complete graph K
n
(n N

), ii) regular
graph R of degree r on n vertices where 1 r n 1, iii) wheel graph
W
m
with m+ 1 vertives, and iv) hypercube graph B
k
where B = {0, 1}
and the natural number k > 1.
4. In a weekend party at CSE faculty, every one shakes hand with each
other. Prove that the number of individuals who shakes hand with an odd
number of participants is an even number.
5. How many simple graphs G are there, provided that it has 5 vertices, and
that it:
a) has none isolated vertex? b) has exactly 2 isolated vertices?
c) has at least 2 isolated vertices?
5.4.2 Problem 2: Representations of Graph Connectivity
1. Write down the adjacency matrix of
a/ the complete graphs K
2
, K
3
, K
4
; b/ the wheel graph W
m
with m = 12;
66 CHAPTER 5. GRAPH THEORY AND MODELS
c/ the regular graph R = (V, E) of degree r = 2 if |V | = 6;
d/ the hypercube graph B
k
where k = 3 and k = 4
2. Write down the incidence matrix and the adjacency matrix of
a/ K
5
, and b/ the complete bipartite graph K
2,3
.
3. Prove that G or G is a connected graph., where G is the complement
graph of a simple graph G.

Remind you that: in a undirected graph G,

* a simple path P is a path consisting no duplicated edges,
* an elementary path is a path that consists no duplicated vertices;
* removing a vertex u implies deleting every edge that has connected with
u;
* a vertex v is called a cut vertex i H := G\ v is disconnected;
* removing an edge e = (u, v) still keeps u, v in G!
* an edge e = (u, v) is called a cut edge or bridge i G\ e is
disconnected.

4. Suppose that G = (V, E) be a simple and undirected graph with n vertices

and m edges. Choose v V with d(v) = k.
i/ In general case, can you nd out the lower bound and upper bound of k?
ii/ Describe the graph H = G\ v := G \
_
{v}, E \ {(v, t) E}
_
by words.
iii/ Take G be the wheel graph with 13 vertices, and choose v be its
center. Can you determine m, k and draw the resulting graph H = G\ v.
5.4. PROBLEMS- SOLVING HINTS 67
5. (*) Let d
0
(G) be the minimal of degrees of the vertices of an n-vertex
graph G.
i) Prove that if d
0
(G)
n1
2
, the graph is connected.
ii) Can
n1
2
be substituted for
n1
2
in the previous statement?
6. (*) Let G = (V, E) be a simple and undirected graph with n vertices, m
edges and p connected components. Prove that
n p m
(n p)(n p + 1)
2
a) Imply that a connected graph with n vertices contains at least n 1
edges.
b) Imply that if 2m > (n 1)(n 2) then G is connected.
7. (*) Let G = (V, E) be a simple and undirected graph with
V = {1, 2, 3, . . . , n}. Call M = [M
ij
] be the adjacency matrix of G, that is
M
ij
=
_
1 if ij E
0 if ij E
Let B = M
k
= [B
ij
] (where k > 0) be the kth-power matrix of M.
Prove that B
ij
is the number of elementary paths of length k from
vertex i to vertex j. What can we say when B
ij
= 1?
Put S = I +M +M
2
+. . . +M
n1
. Prove that G is connected if and
only if S
ij
= 1 for all i, j = 1..n.
5.4.3 Problem 3: Isomorphism Graph Invariants
68 CHAPTER 5. GRAPH THEORY AND MODELS
Graph invariant is a property of graphs that does not change under graph
isomorphisms. Testing whether two graphs are isomorphic, in general is very
dicult; but if you could nd out a graph invariant that does not the same for
two concerned graphs, then you are condent to say the two graphs are not
isomorphic!

1. Choose the best right answer if your friend asks that: Two graphs
G = (V, E) and H = (W, F) are isomorphic when
(A) there exists a bijection f from the vertex set V to the vertex set W?
(B) there exists an injection g from the vertex set V to the vertex set W?
(C) their numbers of connected components are the same?
(D) there exists a bijective mapping f from the vertex set V to the vertex
set W and a bijective mapping g from the edge set E to the edge set F?
(E) None of the above?
Note 1: The numbers of connected components is a graph invariant.
2. Let graphs G and H be isomorphic. Prove that
for every d 0, the number of vertices of degree d in G and H are
the same;
for every l 3, the number of simple cycles of length l in G and H
are the same.
Note 2: The number of vertices of xed degree d and the number of
simple cycles of xed length l in G and H are graph invariants.
In the next question, choose the best right answer if your friend asks that:
5.4. PROBLEMS- SOLVING HINTS 69
3. For two isomorphic graphs, we have:
a. the same number of vertices; b. the same number of edges
c. the same pattern of vertex degrees (after sorting)
d. the same number of connected components
e. Any one of the above options
4. Two graphs G = (V, E) and H = (W, F) are isomorphic when
a. there exists an injective mapping f from the vertex set V to the vertex
set W
b. there exists a bijective mapping f from the vertex set V to the vertex
set W
c. there exists a bijective mapping f from the vertex set V to the vertex
set W and a bijective mapping g from the edge set E to the edge set F
d. there exists a bijective mapping f from the vertex set V to the vertex
set W such that the adjacency relation is preserved, i.e.,
(u, v) E (f(u), f(v)) F, for u, v V
e. None of the above is CORRECT.
5.4.4 Problem 4: Bipartite graphs
1. Prove that: a graph G is bipartite i every circle C of G has an even
length
2. Prove that i) we could arrange rows and columns of B such that B is the
adjacency matrix of an undirected graph G; and ii) we could arrange
vertices such that G is a bipartite graph; where B is the following square
70 CHAPTER 5. GRAPH THEORY AND MODELS
matrix
B =
_

_
0 0 0 0 0 1 0 0 0
0 1 1 0 0 0 0 0 0
0 0 0 0 1 1 0 0 0
1 0 0 1 0 0 0 0 0
0 0 0 0 0 0 1 1 1
1 1 0 0 0 0 0 0 0
0 1 1 0 0 0 0 0 0
0 0 0 0 1 0 1 1 0
0 0 1 0 0 0 0 0 0
_

_
,
3. Choose the best right answer if your friend asks that: For an undirected,
bipartite graph G = (V, E) with adjacency matrix M(G), we have:
a. the vertex set V can be split into two disjoint subsets A, B
b. |E| = |A|.|B|; c. M(G) is a symmetric matrix
d. G contains a Hamilton cycle;
e. two of the four options a. to d. are CORRECT.
5.4.5 Problem 5: Directed graphs

Remind you that: a directed graph (digraph) G = (V, A) consists of vertex set
V and a set A of ordered pairs of elements in V . We call As members the
arcs or directed edges of G. Formally, we understand that for two arcs
a, b A that are dened on two vertices u, v, then we can write
a =

(u, v) = uv, b =

(v, u) = vu A then a = b.
5.4. PROBLEMS- SOLVING HINTS 71
u is the initial (emerging) vertex of a, and v is the terminal (ending) vertex of a.
* A path P of length k in a digraph G = (V, A) is a sequence of distinct vertices
P = v
0
, v
1
, . . . , v
k
where v
i1
v
i
A, i = 1..k.
* A cycle C of length k in G = (V, A) is a path
C = v
0
, v
1
, . . . , v
k
where v
i1
v
i
A, i = 1..k and such that v
0
= v
k
* If there is no cycles in G, then G is called acyclic digraph (or PERT chart).
Remark that PERT chart is employed intensively in task-scheduling problems,
where we want to model ordered lists or structures that have prerequisite
property.

1. Could you provide the denition of the adjacency matrix of a digraph

G = (V, A) with n vertices?
2. Could you provide the denition of the incidence matrix of a digraph
G = (V, A) with n vertices and with m arcs?
3. In a bioscience program at HCMUT in 2010, a student wishes to take 8
courses listed in the below table:
a) Draw the PERT chart G = (V, A) to illustrate the prerequisite relation.
b) Provide the incidence matrix and the adjacency matrix of G
c) Determine all simple paths in G
d) Determine all cycles in G
72 CHAPTER 5. GRAPH THEORY AND MODELS
Course name Prerequisite
Advanced biotech B
Biotech C
Cell biology H
DNA structures C
Enzyme activity D, G
Food science E
Genetic engineering C
Human biology None
Table 5.1: Prerequisite relation between bioscience course units
5.4.6 Problem 6: Trees
1. Which of the following statements is false? The complete bipartite graph
K
m,n
(A) is connected
(B) has spanning trees
(C) can be employed in matching problems
(D) is not isomorphic to the bipartite graph K
m,n
(E) can be used for job assignment problems
2. A tree T has three vertices of degree 3 and four vertices of degree 2. The
remaining vertices all have degree 1.
Choose the best right answer if your friend asks that:
The number of vertices of degree 1 is:
a. 3; b. 4; c. 5; d. 6; e. 7
5.5. ALGORITHMS ON GRAPHS 73
3. Choose the best right answer if your friend asks that: Which one of the
following properties must be WRONG in an arbitrary tree T = (V, E):
a. there is a unique path between every pair of vertices
b. adding any edge creates a cycle
c. removing any adge disconnects the graph
d. if |V | > 1 then T has at least two leaves
e. |V | = |E| 1
4. A forest is an undirected graph (not neccessarily connected) such that
each connected component is a tree. Let G be a forest having n verices
and m connected component, where n, m are natural numbers.
Choose the best right answer if your friend asks that:
Then G has
a. n m edges; b. n m1 edges c. n m+ 1 edges
d. n m+ 2 edges e. n +m edges
5.5 Algorithms on Graphs
5.5.1 Shortest paths and Dijkstra Algorithm
In 1960, the famous computer scientist Edsgar Dijkstra discovered an algorithm
that nds the shortest path between two vertices of a graph, in an amount of
time linear in the number of edges. The algorithm was named after him
(Dijkstras Algorithm), and something like it is used (for example) in Google
Maps and every other piece of software to nd directions.
Theres a famous anecdote where Dijkstra (who worked in a math department,
in which country?) was trying to explain his new result to a colleague. And the
74 CHAPTER 5. GRAPH THEORY AND MODELS
colleague said, but I dont understand. Given any graph, theres at most a
nite number of non-intersecting paths. Every nite set has a minimal element,
so just enumerate them all and youre done. The colleagues argument does,
indeed, show that the shortest path problem is computable. But from a modern
perspective, showing a problem is computable does not say much. The
important question is whether a problem is solvable eciently.
Setting
Let G = (V, E, W) be a positive weighted graph with vertices V , edges E and
with the weight mapping W : E R
+
. Choose two arbitrary vertices a, z V
and let P be the set of all paths P from a to z. Consider the following problem:
Find a path P
0
P such that
W(P
0
) = min{W(P) : P P} where W(P) :=

eP
W(e).
Single source shortest path in connected simple graphs.
Suppose explicitly that G is simple with vertices V = {a = v
1
, v
2
, v
3
, . . . , v
n
},
with weights W(v
i
, v
j
) R
+
whenever (v
i
, v
j
) E, and W(v
i
, v
j
) =
otherwise. Without loss of generality we x vertex a = v
1
, and formulate an
iterative algorithm, due to Edsger W. Dijkstra, to nd a shortest path P
0
from
vertex a = v
1
to any vertex z = a.
For any vertex v, let L(v) be its label, measuring the shortest length from a
to v. At any step, some have temporary labels and the rest have permanent
labels. Of course, at the begining L(a) = 0 (xed or permanent label) and
L(v) = (temporary labels) for v = a.
5.5. ALGORITHMS ON GRAPHS 75
Denote by S V the set of xed label vertices, in the sense that
S = {v V : shortest length from a to v = L(v) < }.
Dijkstras algorithm D(G, a, z) to nd a shortest path P
0
from a = v
1
to any
vertex z = a is based on two basic ideas:
1. Enlarge as much as possible the set S until S consists of z
2. Ensure that whenever we add a vertex u to S then its permanent label
L(u) must give the shortest length from the source a to the considered
vertex u.
Ideas are extracted from the observation that: if P
0
= [a, . . . , w, z] is a shortest
path from a to z then for any vertex w appearing earlier than z in P
0
then:
L(w) < L(z).
We implement ideas by an iterative algorithm, where at each step/iteration
we
1/ update label L(v) of every vertex v S to possible smallest weight (up to
that iteration): be done by employing the rule
L(v) = min{L(v), L(w) +W(w, v)} for v S, w S (w, v) E;
2/ nd the vertex u S that has L(u) minimal, and enlarge S = S {u}: be
done easily since |V \ S| is nite and get smaller after each iteration.
We accept the lemma below, its proof can be found in any standard graph book.
Lemma 20. If L(u) is the permanent label of vertex u then L(u) is the length
of a shortest path from a = v
1
to u.
Dijkstras algorithm to compute shortest path P
0
= [a, . . . , w, z].
76 CHAPTER 5. GRAPH THEORY AND MODELS
Algorithm 1 Computing P
0
= [a, . . . , w, z]
Input G = (V, E, W, a, z), a positive weighted edge, simple, connected graph
with a xed source a, and the sink (target) vertex z
Output L(z) the length of a shortest path and the path P
0
= [a, . . . , w, z] itself
function Singlesource-shortest path( G)
1. Initialization
(1i) Set L(a) = 0, and L(v) = for v = a.
(1ii) Set S = (the set of xed label vertices)
2. Checking whether we arrive at z?
if z S then stop; output L(z).
3. Get next vertex
(3i) choose vertex u V \ S together with the smallest value of L(u).
(3ii) Set S = S {u}
4. Revise labels
For each v S, and there is vertex u S such that (u, v) E, (v connects
u), set
L(v) = min{L(v), L(u) +W(u, v)}
If L(z) < then S = S {z}. Go back line 2.
end function
Example
Convention: we circle a xed vertex u (the uncircled vertices have temporary
labels) when the algorithm proceeds. Whenever circling a xed vertex v, that
vertex has permanent label and does not change value L(v) at all; we will also
label it with the name of the vertex from which it was labeled. This operation
allows us to track back the full path P
0
to the source a from the sink when the
algorithm stops. Hence, at Line 2, when z S, return L(z) and use the names
at labels of vertices to provide the explicit path P
0
.
We change names of vertices v
1
, v
2
, . . . , v
6
to {a, b, c, d, e, z} respectively.
Initialization.
Set L(a) = 0, circle a, and set L(v) = for v = a.
Set S = [] (empty list): no circled vertex has been found yet.
5.5. ALGORITHMS ON GRAPHS 77
(G)
b=
a=
c = v3
v1
v2
d=v4
e=v5
z=v6
2
7
1
4
1
2
2
Figure 5.2: Shortest path in a weighted graph G
Iteration 1:
Line 2: Checking z S. No, z still is not circled.
Line 3: Get next vertex. We select u V \ S = V together with the smallest
value of L(u): only a is the candidate. Update S = S {a} = [a], cirle a.
Line 4: Revise labels of vertices v S connecting to the last xed vertex in S.
We label the vertices b and d adjacent to a S, obtain
L(b) = min{L(b), L(a) +W(a, b)} = min{, 0 + 2} = 2, assign (a, 2) to b
L(d) = min{L(d), L(a) +W(a, d)} = min{, 0 + 1} = 1, assign (a, 1) to d
Iteration 2:
Line 2: Checking z S. No, z still is not circled.
Line 3: Get next vertex. We select u V \ S (or equivalently u S) having the
smallest value L(u): d is the candidate since L(d) < L(b). Update
S = S {d} = [a, d], cirle d
Line 4: Revise labels of vertices v S that connect to the last xed vertex in S.
78 CHAPTER 5. GRAPH THEORY AND MODELS
We label the vertex e adjacent to d, obtain the new label
L(e) = min{L(e), L(d) +W(d, e)} = min{, 1 + 4} = 5, assign (d, 5) to e
Iteration 3:
Line 2: Checking z S. No, z still is not circled.
Line 3: Get next vertex. For all u S choose the one having label smallest: b is
the right candidate since L(b) = 2 < 5 = L(e). Update S = S {b} = [a, d, b],
cirle b.
Line 4: Revise labels of v S that connect to the last xed vertex in S. The
vertices c, e both are adjacent to b, so we obtain the new labels
L(c) = min{L(c), L(b) +W(b, c)} = min{, 2 + 7} = 9, assign (b, 9) to c
L(e) = min{L(e), L(b) +W(b, e)} = min{5, 2 + 1} = 3, assign (b, 3) to e
Note that e has been just updated its label from (d, 5) to (b, 3), a better
improvement!
Iteration 4:
Line 2: Checking z S. No, z still is not circled.
Line 3: Get next vertex. For u S having L(u) smallest, then u = e is the best
candidate since L(e) = 3 < 9 = L(c). Update S = S {e} = [a, d, b, e], cirle e.
Line 4: Revise labels of v S and connect to the last xed vertex in S. The
vertex z now is adjacent to e, obtain the new label
L(z) = min{L(z), L(e) +W(e, z)} = min{, 3 + 2} = 5, assign (e, 5) to z.
Update S = S {z}. Go back line 2, stop. The explicit path P
0
= [a, b, e, z]
5.5. ALGORITHMS ON GRAPHS 79
from a to z is listed from the inverse trail of labels:
[z (e, 5) (b, 3) (a, 2)], the shotrest length is L(z) = 5.
Matrix form of Dijkstras algorithm. When you understand the algorithm,
you can nd the answer by using the following Table 5.5.1.
Step Vertex to Uncircled
be circled a b c d e z vertices
1 a 0 2 1 b, d, c, e, z
2 d 0 2 1 5 b, c, e, z
3 b 0 2 9 1 3 c, e, z
4 e 0 2 9 1 3 5 c, z
5 (nal) z 0 2 7 1 3 5 c
6 (extra) c 0 2 7 1 3 5
Table 5.2: Step by step runing of Dijkstras algorithm
At Step 5, c has been just updated its label from (b, 9) to (z, 7), a better
improvement!.
Note 1: if you want to nd shortest paths from source a to every vertex in
graph, then you proceed computing the above table until no vertex occurs in the
nal column. If this is the case, then the nal row of the mentioned table
provides all shortest path from a to every vertex in graph.
Complexity analysis of Dijkstras algorithm
Theorem 21. Let G = (V, E, W) be a positive weighted edge, simple, connected
graph having n vertices. If f(n) is the number of times Algorithm 5.5.1
examines an edge of G in the worst case, then f(n) = O(n
2
).
5.5.2 Max-ow Min-cut algorithm by Ford-Fulkerson
Go to library, prepare for your presentation.
80 CHAPTER 5. GRAPH THEORY AND MODELS
Chapter 6
Algebraic Model and
Polynomial Equations
The structure of this chapter
1/ Gauss Elimination Its Variations
2/ Nonlinear Polynomial Equations
3/ Modeling by Algebraic Methods

6.1 Linear polynomial equations

- Most actual dependencies are nonlinear
- Linearity assumptions are widely spread, since usually the nonlinearity of the
analyzed phenomenon does not have great impact on the results
81
82 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
6.1.1 Gauss Elimination - Contents
The object: a linear system of equations, written A.x = b in which
A R
mn
; b R
m
, vector of unknowns x = (x
1
, x
2
, , x
n
).
This brief form stands for the below system
b
i
=
n

j=1
a
ij
x
j
, i = 1..m
where the coecients a
i,j
, b
i
are known real numbers, and the coecient matrix
A
mn
=
_

_
a
1,1
a
1,2
a
1,3
. . . a
1,n1
a
1,n
a
2,1
a
2,2
a
2,3
. . . a
2,n1
a
2,n
a
3,1
a
3,2
a
3,3
. . . a
3,n1
a
3,n
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
m,1
a
m,2
a
m,3
. . . a
m,n1
a
m,n
_

_
. (6.1)
The aim: discuss numerical methods for nding unknowns x = (x
1
, x
2
, , x
n
)
We will deal with the three cases:

(i) The case where m = n: the number of equations and the number of
unknowns are equal so that the coecient matrix A
mn
is square.
(ii) The case where m < n so that we might have to nd the minimum- norm
solution among the numerous solutions.
(iii) The case where m > n (the number of equations is greater than the
number of unknowns) so that there might exist no exact solution and we must
nd a solution based on global error minimization, like the LSE (Least-squares
error) solution.
6.1. LINEAR POLYNOMIAL EQUATIONS 83

Nonsingular means invertible. Given A R

nn
. If there exists a second
matrix B R
nn
such that AB = BA = I
then
we say B is the inverse of A, write B = A
1
, and
say A is nonsingular.
Otherwise, when no such matrix B exists, A is singular.
Theorem 22. Given a square matrix A R
nn
, the followings are equivalent:
1. A is nonsingular
2. The columns of A form an independent set of vectors
3. The rows of A form an independent set of vectors
4. The linear system A.x = b has a unique solution for all vectors b R
n
5. The homogenous system A.x = 0 has only the trivial solution x = 0
6. The determinant det(A) = 0
(i) The nonsingular case (m = n). We obtain the unique solution x = A
1
b
when A is nonsingular, that is det(A) = 0 or rank(A) = n.
(ii) The underdetermined case (m < n): Minimum-Norm Solution.
(iii) The Overdetermined Case (m > n): Least Square Error (LSE) Solution

84 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS

(ii) The second case of the system A.x = b.
If the number of equations is less than the number of unknowns, m < n, the
solution is not unique, but numerous. Suppose the m rows of the coecient
matrix A are independent. Then, any n -dimensional vector can be decomposed
into two components
x = x
+
+x

Range(A) + Ker(A)
where the one x
+
is in the row space Range(A) of A, that can be
expressed as a linear combination of the m row vectors:
x
+
= A
T
,
and the other x

is in the null space Ker(A) being orthogonal to the row

space so that
Ax

= 0.
Then
A(x
+
+x

) = AA
T
+Ax

= AA
T
+ 0 = AA
T
= b
Since AA
T
is supposedly a nonsingular mm matrix resulting from
multiplying an mn matrix by an n m matrix, we can solve the equation
AA
T
= b for to get

o
= [AA
T
]
1
b
Therefore,
x
+

= A
T

o
= A
T
[AA
T
]
1
b (6.2)
provides a solution for A.x = b, but not unique, due to many possibilities of x

!
6.1. LINEAR POLYNOMIAL EQUATIONS 85
Denition 23. The matrix A
T
[AA
T
]
1
is called the right pseudo- (generalized)
inverse of A, denoted pinv(A).

Based on the principle that any one of the two perpendicular legs is shorter
than the hypotenuse in a right-angled triangle, Eq. (6.2) is believed to represent
the minimum-norm solution.
MATLAB has the pinv() command for obtaining the pseudo-inverse. We can
use this command or
the slash(/) operator to nd the minimum-norm solution (6.2)
to the system of linear equations
A.x = b, where A R
mn
; b R
n
. (6.3)

Remark. The solution (6.2)

x
+

= A
T

o
= A
T
[AA
T
]
1
b
can be viewed as the projection of an arbitrary solution x

onto the row space

Range(A) of the coef[U+FB01]cient matrix A spanned by the row vectors.
The remaining component x

of the solution x

= x

x
+

= [Id pinv(A)A]x

Ker(A)
The matrix P = [Id pinv(A)A] is called the projection operator.

86 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS

(iii) The Overdetermined Case (m > n): Least Square Error (LSE) Solution
If the number of equations is greater than the number of unknowns, m > n
there exists no solution satisfying all the equations strictly. Thus we try to
nd the LSE (least-squares error) solution minimizing the norm of the
(inevitable) error vector
e = Ax b
Then, our problem is to minimize the objective function
J =
1
2
||e||
2
=
1
2
||Ax b||
2
=
1
2
[Ax b]
T
[Ax b] (6.4)
whose (LSE) solution can be obtained by setting the derivative of this function
with respect to x to zero.

x
= A
T
[Ax b] = 0 x

= [A
T
A]
1
A
T
b
x

= [A
T
A]
1
A
T
b (6.5)
Denition 24. The matrix [A
T
A]
1
A
T
is called the left pseudo- inverse of A,
also denoted pinv(A).
The LSE solution can be obtained by using
the MATLABs pinv() command or
the backslash () operator
6.1. LINEAR POLYNOMIAL EQUATIONS 87
6.1.2 Gauss Elimination- Technique
Consider a linear system
A.x = b, where A R
mn
; b R
n
...()
Denition 25. Elementary row operations or row reductions include:
a/ Multiply both sides of an equation by a non-zero constant
b/ Add a multiple of one equation to another
c/ Interchange two equations
We now form the augmented matrix A

= [A|b],
then using row reductions on A

to obtain a row equivalent matrix

R = [U|c]
and its row equivalent system
U.x = c ().
Two systems (*) and (**) have the same solution sets.
Gauss Elimination means systematically eliminating nonzero elements below
the main diagonal of the coecient matrix A:
A

rowreductions

R such that R = [U|c],

where U is upper triangular, and we have rank(U) = rank(A).
Gauss Elimination, hence consists of two steps
88 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
1. Row reduction: to convert (*) to an equivalent upper triangular U (using
row reduction), having the form:
U.x = c, easier to solve, and
2. Backward substitution: solve in reverse order,
How do we nd the matrix U? Let look at a specic example.
Example 11. Consider the linear system
_

_
4x
1
+ 2x
2
x
3
= 5
x
1
+ 4x
2
+x
3
= 12
2x
1
x
2
+ 4x
3
= 12
in the form A.x = b:
_

_
4 2 1
1 4 1
2 1 4
_

_
_

_
x
1
x
2
x
3
_

_
=
_

_
5
12
12
_

_
(6.6)
The augmented matrix
A

=
_

_
4 2 1 5
1 4 1 12
2 1 4 12
_

_
4 2 1 5
0 7/2 5/4 43/4
0 0 73/14 219/14
_

_
= R
This augmented matrix represents a triangular system (the coecient matrix is
triangular):
R = [U|c] U.x = c, i.e.
6.1. LINEAR POLYNOMIAL EQUATIONS 89
_

_
4 2 1
0 7/2 5/4
0 0 73/14
_

_
_

_
x
1
x
2
x
3
_

_
=
_

_
5
43/4
219/14
_

_
; (6.7)
Backward substitution means using:
the third row: 73/14x
3
= 219/14 x
3
= 3
the second row: 7/2x
2
+ 5/4x
3
= 43/4 x
2
= 2
the rst row: 4x
1
+ 2x
2
x
3
= 5 x
1
= 1.
The solution is x = (1, 2, 3).

6.1.3 Matlab section

1. Row-echelon form: a matrix derived from another by a nite sequence of
elementary operations,
2. gauss(A, b): performs Gaussian elimination on the system
A.x = b;
returns the upper triangular factor U (a row-echelon form), and we have
rank(U) = rank(A).
3. rref([A b]) = R: Row reduced Echelon Form of [A|b], performs the Gauss
elimination on [A|b], called the extended matrix, returns the unique
reduced row echelon form R
We obtain the unique solution x = A
1
b
when A is nonsingular, that is det(A) = 0 or rank(A) = n.
90 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
4. BackwardSubstitute(R)
A row echelon form has the properties:
a/ all the nonzero rows precede all the zero rows, and the rst nonzero
element in a nonzero row appears to the right of the rst nonzero element in the
preceding row;.
b/ in the rst nonzero element in each nonzero row is 1 and is the only
nonzero element in its column
Summary command in MATLAB.

Quantity MATLAB Syntax Comment

pseudo- inverse of A pinv(A)
or generalized-inverse of A pinv(A)
minimum-norm solution x = A/b or
x = pinv(A) b
LSE solution x = Ab
or x = pinv(A) b

6.2. SYMBOLIC COMPUTATIONFOR FINDINGROOTS OF NONLINEAR EQUATIONS91
6.1.4 Practical projects (Optional)
1/ (Max 3 student group) Study Leonchiev model in Economics using linear
systems.
[Vasily Leonchiev, an American economist who won the Nobel Prize, private
capitalism must account for one-half of the Gross National Product...]
2/ (Max 3 student group) Study PageRank engine
6.2 Symbolic computation for nding roots of
nonlinear equations
This chapter presents selective topics of Computer Algebra (C.A.). We
specically intergrate well known algebraic techniques such as the Groebner
Basis Method to some new applicable elds growing very fast in recent years.
Applicable elds range from pure math, industrial statistics to statistical
modelling, mathematical genomics, bioinformatics among others.
Benets of symbolic computation
Computer algebra systems (CAS) or symbolic computation packages provide
many benets:
1. dependability of mathematical formulas is increased through new
ways to produce or check the results exactly;
2. dependability and precision of numerical results is enhanced
through unrestricted-precision integer, rational, real and complex
arithmetic;
92 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
3. models is formulated precisely through using operators and high level
mathematical objects; also by the implementation of exact algorithms
which havent been done in numerical packages;
4. handling combinatorial structures that involve lengthy iterations and
recursions, where computational intensity can limit the models;
5. due to the powerful abstraction of symbols, we are able to represent
many complex processes in various areas such as: biometrics,
computational physics and chemistry, control systems . . .
Grobner basis method
The core of computer algebra systems is the Groebner Basis Method. But why
is Computer Algebra? At least three reasons to study:
1/ symbolic solution or numerical solution: symbolic-analytic solutions are more
demanded in practical usages nowadays
2/ analyzing and modelling are important than computing
3/ the role of algebraic computation combined with geometric, theorectical
group computation becomes increasingly important, especially in Math and Stat
modeling; eg., determining good-genes and bad-genes in human-being genome,
measuring risk in nancial investment, the combinatorial aution problem in
e-commerce ...
Solve a non-linear system, you must use the Groebner package! Type in Maple
with(Groebner).
Example 12. Let f := x
2
+y +z 1; g := x +y
2
+z 1; h := x +y +z
2
1;
and J :=< f, g, h >. Then
6.3. ALGEBRAIC MODELING 93
G := Basis(J, plex(x, y, z)); get
G := [z
6
+4 z
3
4 z
4
z
2
, 2 z
2
y +z
4
z
2
, y
2
z
2
+z y, x +y +z
2
1];
Check Z(J) = Z(G) a nite set, use:
IsZeroDimensional(G); true
and to solve this, use:
solve(G, {x, z, y});
The root set Z(J) is
{{y = 0, z = 0, x = 1}, {x = 0, y = 1, z = 0}, {z = 1, x = 0, y = 0}, {z = 1, x =
0, y = 0}, {x = RootOf(Z
2
+ 2Z 1, label = ...), y = x; z = x}}.
See more at http://www.cse.hcmut.edu.vn/mnguyen/
classes/ACA/Computational-Algebraic-Geometry-2010.html
6.3 Algebraic Modeling
6.3.1 In Industrial Statistics
Introduction to Design of Experiments (DOE).
Design of Experiments (DOE) is a mathematics branch studying
statistically designed experiments. But why DOE?
1930s of the 20-th century, when Sir R. A. Fisher used Latin squares to
randomize the plant varieties
R.C. Bose in the 1950s in India and then in the US: the mathematical
theory of combinatorial designs
Nowadays, DOE is extensively studied and employed in , and the
Mathematics for DOE is very rich. Mostly studied directions are:
94 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
study of Factorial designs: mathematically described as matrices
which consist many factors, each has several settings (levels), and
these settings are arranged in a regular way.
The main tools are combinatorics, algebra, geometry.
study of Optimal designs
study of Computer experiments ...
Modern industrial organisations in manufacturing and services are subject
to increasing competitive pressures and rising customer expectations. Two
key aims: a/ to satisfy and delight various customers while simultaneously
b/ improving eciencies and cutting costs.
A systematic approach to the evaluation of benets from process
improvement and quality by design (QbD) that can be implemented
within and across organisations, be summarised and classied using a
four-step quality ladder:
(1) re ghting , (2) inspection,
(3) process control, and (4) quality by design.
Statistical methods corresponding to these steps are:
(a) Data Accumulation, (b) Sampling,
(c) Statistical Process Control and (d) DOE!
DOE covers many smaller topics: robust designs, optimal designs,
combinatorial designs, fractional factorial designs. DOE helps us to
explore the relationships between the key input process variables (or
factors) and theoutput performance characteristics (responses or quality
characteristics) Hence, the whole picture includes operations, machines,
6.3. ALGEBRAIC MODELING 95
methods, people and other resources that transform some input into an
output characterised by one or more response variables.
Mathematical techniques for studying DOE are very rich:
combinatorics, graphs, group theory, nite elds, coding theory,
nite geometry and algebra, mathematical programming
Computational Algebaric Geometry & Commutative Algebra.
Applied to many areas nowadays: agriculture, bio-informatics
[microarray], medicine and pharmaceutics, automobile engineering.
What are factorial designs and fractional factorial designs?
Factorial designs. Suppose that we have n nite sets Q
1
, Q
2
, . . . , Q
n
contained
in a eld k, called the factor sets or factors. We use a few notation below:
The (full) factorial design with respect to these n factors is the Cartesian
product D = Q
1
. . . Q
n
k
n
.
A design point p = (p
1
, . . . , p
n
) is an element of D.
Moreover, r
i
:= |Q
i
| is the number of levels of the factor i.
We say that D is symmetric if r
i
= r for all i; otherwise, D is mixed, that
is r
i
= r
j
for some i = j.
Let s
1
, s
2
, . . . , s
m
(m n) be the distinct levels of D, and suppose that D has
exactly a
i
factors with s
i
levels. We call s
a
1
1
s
a
2
2
s
a
m
m
the design type of D.
For example, if Q
1
= {0, 1, 2, 3}, Q
2
= Q
3
= Q
4
= {0, 1}, then
D = {(0, 0, 0, 0), (0, 0, 0, 1), (0, 0, 1, 0), (0, 0, 1, 1), . . . , (3, 1, 1, 0), (3, 1, 1, 1)}
96 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
is the 4 2
3
mixed factorial design.
Fractional factorial designs. Suppose D is a s
a
1
1
s
a
m
m
mixed factorial design.
A fraction F of D is a subset consisting of elements of D. If F has an element
with multiplicity greater than one, we say F has replications. This is also called
an s
a
1
1
s
a
m
m
fractional design. For example
F = { (0, 0, 0, 0), (0, 1, 0, 1), (0, 0, 1, 1), (0, 1, 1, 0), (1, 0, 0, 0), (1, 1, 0, 1),
(1, 0, 1, 1), (1, 1, 1, 0), (2, 1, 1, 1), (2, 0, 1, 0), (2, 1, 0, 0), (2, 0, 0, 1),
(3, 1, 0, 0), (3, 1, 0, 1), (3, 1, 1, 0), (3, 1, 1, 1) }
is a 4 2
3
mixed fractional design. We usually consider a fractional design as a
matrix whose rows correspond to the elements of the multiset, in any order, and
whose columns correspond to the factors. So the example above becomes
F =
_

_
0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3
0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0
0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0
0 1 1 0 0 1 1 0 1 0 0 1 1 0 0 1
_

_
T
,
where T denotes transpose.
Example 13. Fix n = 3, we use the full factorial design 2
3
with three
binary factors to nd the relationship between the factor x
1
of mixture ratio,
the factor x
2
of temperature, the factor x
3
of experiment time period and the
response y of wood toughness.
The levels of factors are given in Table 6.1, and simulated experiments in Table
6.2:
6.3. ALGEBRAIC MODELING 97
Factor Low (0) High (1)
Mix(ture) Ratio 45p 55p
Temp(erature) 1000C 1500C
Time period 30min 90min
Table 6.1: Factor levels of 2
3
factorial experiment
RUN Mix Ratio Temp Time Response Y
1 45p (-) 100C (-) 30m (-) 8
2 55p (+) 100C (-) 30m (-) 9
3 45p (-) 150C (+) 30m (-) 34
4 55p (+) 150C (+) 30m (-) 52
5 45p (-) 100C (-) 90m (+) 16
6 55p (+) 100C (-) 90m (+) 22
7 45p (-) 150C (+) 90m (+) 45
8 55p (+) 150C (+) 90m (+) 56
Table 6.2: Results of an example 2
3
Factorial Experiment
The linear model for the wood toughness y, t by the eight experimental runs, is
y = f(x
1
, x
2
, x
3
) =
0
+
1
x
1
+
2
x
2
+
3
x
3
+
12
x
1
x
2
+
13
x
1
x
3
+
23
x
2
x
3
+
123
x
1
x
2
x
3
.
Example 14. A 2
3
fractional factorial design has only 4 runs, whose
columns correspond to the three binary factors, of the full 2
3
-factorial design of
8 runs above.
This fraction is extracted from the below Hadamard matrix of order 4 (in which
every pair of rows and pair of columns have their inner product equals to 0).
This matrix provides enough information to estimate main eects of the mixture
98 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
ratio x
1
, the temperature x
2
, the experiment time period x
3
in Example 1:
F =
_

_
1 x
1
x
2
x
3

1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1
_

_
T
.
But why using Fractional Factorial Designs? To cut cost in scientic
investigations and/or in industrial manufacturing. In particular case here, if we
employ the four experimental runs of F, we can t only a linear model for the
response y of wood toughness, of the form:
y = f(x
1
, x
2
, x
3
) =
0
+
1
x
1
+
2
x
2
+
3
x
3
.
So using less resource, we now could t a model that consists of only main
factor eects!
Assignment. Write a Maple program or Singular program to compute
a/ the full design D = Q
1
. . . Q
4
above, where Q
1
= {0, 1, 2, 3},
Q
2
= Q
3
= Q
4
= {0, 1};
b/ the mixed 4 2
3
fractional design F of 16 rows above.
Three Fundamental Research Problems on FFD:
1. Problem 1: Constructing a fraction with given estimable terms (masters
6.3. ALGEBRAIC MODELING 99
Hnh 6.1: th G gm cc khu vc v lin kt
level)
2. Problem 2: Constructing a strength t balanced fraction with given
estimable terms (PhD level!)
3. Problem 3: (PhD level!) Construction of new strength t fractions F,
given
a fraction F
0
of N runs, d factors, with strength t.
We also name this the OA Column Extension Problem:
extend F
0
with a new factor X to have a new fraction F = [F
0
|X] having
the same strength t and d + 1 factors.
6.3.2 In Business Intelligence
Mt cng ty A mun pht trin kinh doanh k loi dch v, nh l cung cp thc
phm, hng ni tht, thit b truyn thng ... ti 4 khu vc chnh TPHCM, k
hiu l I, II, III, IV; bit 4 k 2. S biu din cc khu vc ny v cc i
l kt ni chng cho bi th G nh trn.
Cng ty A mun xy dng cc siu th ti cc khu vc chnh TPHCM ni trn
sao cho tha ng thi hai rng buc:
a/ ch mt siu th c xy ti 1 khu vc, v
100 CHAPTER 6. ALGEBRAIC MODEL AND POLYNOMIAL EQUATIONS
b/ hai khu vc k nhau (c i l kt ni) bt k phi c hai siu th cung
cp 2 dch v khc loi nhau.
Phng n kh thi l phng n xy dng cc siu th ti 4 khu vc chnh ni
trn m tha mn ng thi hai iu kin a/ v b/.
i) C phng n kh thi khng khi k = 2?
ii) Anh (Ch) hy trnh by cc bc c th m hnh ha i s bi ton trn
khi 2 k n, bit 2 n N l s lng khu vc chnh TPHCM.
iii) Khi s loi dch v k = 3, v n = 4 nh cho trong G, hy cho bit c bao
nhiu phng n kh thi.
Chng 7
Probabilistic Models
7.1 Practical Motivation
Probabilistic Methods help us to solve typical problems in daily life as follows.
1. Given the probability that a randomly selected student in a class is a
female is 56%, how much chance that the selected student is a male?
2. The mass, X kg, of silicon produced in a manufacturing process is
modelled by the probability density function (pdf)
f
X
(x) =
3
32
(4x x
2
) if 0 x 4; f
X
(x) = 0 otherwise.
What is the mean of the mass of silicon produced?
3. Eggs sold at a market are usually packaged in boxes of six. The number,
X, of broken eggs in a box has the probability distribution given in the
following table:
101
102 CHNG 7. PROBABILISTIC MODELS
x 0 1 2 3 4 5 6
P[X = x] 0.80 0.14 0.03 0.02 0.01 0 0
Denote by Y the number of unbroken eggs in a box.
What are the average of X and of Y respectively?
4. A discrete random variable X represents the number of trafic accidents
each year at HCMC. Suppose X has the probability mass function
f
j
= P(X = j) =
3
4
_
1
4
_
j
, j = 0, 1, 2, 3, . . . ..
with unit of 100 accidents. Up to how much chance that at most 200
accidents take place in a year?
5. Consider a Markov chain M describing the loyalty of customers to three
retailers Coop, BigC and Walmart, coded by states 0, 1, and 2 respectively.
The transition probability matrix P, obtained in a recent poll, is given by
P =
_
0.4 0.2 0.4
0.6 0 0.4
0.2 0.5 0.3
_
What is the probability in the longrun that the chain M is in state 1, that
is how much chance the customers will go with BigC?
7.2. CONCEPTS AND DEFINITIONS 103
7.2 Concepts and Definitions
Probability is the science of problem-solving in the presence of variability and
uncertainty. We will need the below basic concepts.
Experiments An experiment E is a specific trial/activity (of scientists, human
being) whose outcomes possess randomness. Simple examples are:
Coin throwing- throw a coin, random outcomes are head (H) or tail (T)
Temperature measurement- observe continuously temperatures at noon in
HCMC in 10 days of Summer 2007, random outcomes are the list
[34, 29, 28, 32, 31, 32, 30, 31, 30, 33] (in Celcius degree).
Sample space.
1. Sample space S- set of all possible outcomes.
Example 1: Coin throwing S = {H, T}
2. Events- is subset A of sample space S: A S. Usually we include all
events into a set, called the event set Q := {A : A S and is an event}.
When an experiment E is performed and an outcome a is observed we say
that event A has occurred if a A.
3. Probability distribution (probability function)- a map P from Q to the
interval [0, 1]:
P : Q [0, 1],
A Q P(A) = Prob(A) =
probability or chance that the event A occurs.
104 CHNG 7. PROBABILISTIC MODELS
Axioms of Probability Theory (A. Kolmogorov, 1933).

A1. Probabilities are nonnegative, 0 P(A) 1, where P(A) = Prob(A).

A2. The sample space S has probability 1, that is P(S) = 1
A3. Probabilities of disjoint events A, B, A B = :
P(A B) = P(A or B) = P(A) +P(B),
in which A, B S are events, the sample space S is formed from a specific
random experiment E.
Computing Rule. For finite sample spaces, we assume S = {s
1
, s
2
, , s
n
},
define p
i
= P(s
i
) then
p
i
0, and
n

i=1
p
i
= 1.
Fact 2. If all outcomes have equal probabilities, then
P(A) = Prob(A) =
n
A
n
, where n
A
= |A|.
Example 15. On a single toss of a die, we get only one of six possible
outcomes: 1,2,3,4,5 or 6; then the sample space S = {1, 2, 3, 4, 5, 6}, and
p
i
= P(i) = 1/6, for all i = 1..6
7.2.1 Random Variable and Probability Distribution
Definition 26. A random variable X is a function from a set - sample space S
to the reals R. For any b R, the preimage
A := X
1
(b) = {w : X(w) = b} S
7.2. CONCEPTS AND DEFINITIONS 105
is an event, we understand
Prob{X = b} := Prob(A) =

wA
Prob(w).
For finite set - sample space S then obviously
Prob{X = b} := Prob(A) =
|A|
|S|
.
The probability distribution of a random variable describes how probabilities are
distributed over the (range) values of the random variable.
Expectation and Variance- the Discrete Case
Expectation. The expectation operator defines the expected value (or average
behavior) of a random variable X as
E(X) =

xRange(X)
P(X = x) x, where (7.1)
P(X = x) = P(X
1
(x)); and X
1
(x) = {w : X(w) = x} S.
In words, E(X) is a weighted average of the possible values of X, where the
weight given to a value x is equal to the probability P(X = x) that X assumes
that value.
Note: since the r.v. X : S R is an assignment of values to the points in
sample space S, you could also think
E(X) =

wS
P(w) X(w) equivalently.
106 CHNG 7. PROBABILISTIC MODELS
Variance of a random variable X is
Var(X) = E[(X E(X))
2
].
7.2.2 Why study Probability Distributions?
Citibank in HCMC makes available financial services, including checking and
saving accounts, loans, mortgages, insurance and investment services. These
complicated activities have been done through a Citibanking system consisting
of many modules, like ATMs, or more advanced, the Card Banking Centers
(CBCs).
What would be the services available at CBSs? and How? Each CBC operates as
a waiting line system with randomly arriving customers seeking service at one of
the ATMs. CBC capacity studies are used
- to analyze customer waiting line and
- to determine whether additional ATMs are needed.
Data collected by Citibank showed that the random customer arrivals followed
a probability distribution known as the Poisson distribution. Using the
Poisson distribution, Citibank can compute probabilities for the number of
customers arriving at a CBC during any time period and decisions concerning
the number of ATMs needed.
Discrete probability distributions, such as the one used by Citibank are the topic
of the next section.
7.3. DISCRETE PROBABILITY DISTRIBUTIONS 107
7.3 Discrete Probability Distributions
Definition 27. Discrete random variable X is the one that has a finite range
set. For a discrete ran. var. X, its probability distribution is the probability
function
f(x) = P({X = x},
providing the probability that the r. v. X receives a particular value
x Range(X).
The discrete probability distribution f(x) must fulfill:
f(x) 0, and

xRange(X)
f(x) = 1.
Example 16 (Die tossing). On a single toss of a die, we get only one of six
possible outcomes 1,2,3,4,5 or 6; then the sample space S = {1, 2, 3, 4, 5, 6}.
Define the random variable X : S R
+
to be the identity function Id, that is
X(i) = Id(i) = i, i S.
The probability distribution associated with X is the probability function
f(i) = Prob{X = i} = P{X
1
(i)} =
|X
1
(i)|
|S|
= 1/6, for i = 1..6
Often for discrete distributions, we write p
i
= f(i) = P(i).
Example 17 (Fair bet). A bet whose expected winnings equals to 0 is called a
fair bet. Let the random variable X denote the amount that we win when we
make a certain bet. Find E(X) if there is a 60% chance that we lose 1 USD, a
20% chance that we win 1 USD, and a 20% chance that we win 2 USD. Is this a
fair bet?
108 CHNG 7. PROBABILISTIC MODELS
Lemma 28. For constants a and b, and for a discrete random variable X, we
have
E(aX +b) = a E(X) +b.
Chng minh. Use the definition 7.1.
* Important discrete distributions include Bernoulli, Binomial and Poisson.
1/ Bernoulli Distribution B(p) describes a random variable that can take
only two possible values, i.e. X = {0, 1}. Its probability function is given by
p(1) = P(X = 1) = p, p(0) = P(X = 0) = 1 p for some p [0, 1].
Use Eq. 7.1, it is easy to check that E(X) = p, Var(X) = p(1 p).

2/ Binomial distribution Bin(n, p) describes a random variable X that is a

number of successes in n independent Bernoulli trials with probability of success
p. In other words, X is a sum of n independent Bernoulli r.v. Therefore, X takes
values in {0, 1, ..., n} and its distribution is given by the probability function
p(k) = P(X = k) =
_
n
k
_
p
k
(1 p)
nk
. (7.2)
It is easy to check that E(X) = np, Var(X) = np(1 p).

Proof of Eq. 7.2. Let H and T be two outcomes of an experiment as Coin

throwing, with sample space S
Coin
= {H, T}, and in general, the occurrence
likelihoods P(H) = P({H}) = p; P(T) = 1 p.
7.3. DISCRETE PROBABILITY DISTRIBUTIONS 109
Assume that we perform n trials, called Bernoulli Trials, of the experiment and
each trial is independent of the others. For example, the event H on the first
trial is independent from the event H on the second trial. So both events
have probability p. The sample space S now can be represented by
S = {x
1
x
2
x
n
| x
i
S
Coin
}.
Since the trials are independent, we assign probabilities to the points in S by
P(x
1
x
2
x
n
) = P(x
1
) P(x
2
) P(x
n
).
Our question now is: what is the probability of exactly k successes in n trials of
a binomial experiment where P(success) = p and P(failure) = 1 p?
Let X = Bin(n, p) be the sum of n independent Bernoulli r.v., then X takes
values in {0, 1, ..., n}. The answer, therefore is X = k {0, 1, ..., n}, that means
exactly k successes in n trials. By combinatorial reasoning, the binomial
distribution Bin(n, k) is given by a probability function
p(k) = P(X = k) =
_
n
k
_
p
k
(1 p)
nk
.

Reminder: if V : S R is an assignment of values to the points in sample space

S, then
E(V ) =

wS
P(w) V (w).
110 CHNG 7. PROBABILISTIC MODELS
3/ Poisson distribution: The Poisson distribution is given by
p(x) =
e

x
x!
x = 0, 1, 2, ... (7.3)
where
x = designated number of successes,
> 0 a constant = the average number of successes per unit of time period
e = 2.71 the natural base.
The Poisson distributions mean and the variance are
= ;
2
= = .
7.4 Problems
Problem 1: Describe in details some examples of when we can apply the Poisson
distribution.
Help: Poisson distributions can be used in the followings.
To model of the number of occurrences of some event/ phenomenon in the
time interval (0, t], we can use Formula 7.3. Now x = 0 implies that there
are no occurrences of the event in (0, t], and Prob(x = 0) = p(0) = e

.
To model the number of defects or non-conformities that occur in a
unit of product (unit area, volume ...) say, a semiconductor device, by a
Poisson distribution.
Problem 2: given that the number of wire-bonding defects per unit X is Poisson
distributed with parameter = 4. Compute the probability that a randomly
selected semiconductor device will contain two or fewers wire-bonding defects.
7.4. PROBLEMS 111
Problem 3. Past experience indicates that an average number of 6 customers per
hour stop for banking services at a bank.
(a) Find the probability of 3 customers stopping in any hour.
(b) Find the probability of 3 customers or less stopping in any hour
(c) What is the expected value, or mean, and standard deviation for this
distribution?
Problem 4. Two fair dice are tossed. If the total is 7, we win $100; if the total is
2 or 12, we lose $100; otherwise we lose $10. What is the expected value of the
game?
Problem 5. Known that 1% of the lightbulbs produced by DienQuang Cor. are
defective. Find the probability that 1 bulb is defective in a random sample of 30
bulbs, using
a) the binomial and
b) the Poisson approaximation of the binomial distribution?

112 CHNG 7. PROBABILISTIC MODELS

Course Review. Try the followings.
1. Given a digraph G = (V, A) where V = {a, b, c, d} and A = {ab, bc, bd, dc}.
Draw G, compute its adjacency matrix M and the reachability matrix M

.
2. The same request as above for H = (W, A) where W = {v
1
, v
2
, v
3
, v
4
, v
5
}
and A = {v
1
v
2
, v
2
v
3
, v
3
v
1
, v
3
v
4
, v
5
v
1
, v
5
v
3
}.
3. State the definition of weight matrix W of a weighted digraph G = (V, A).
4. Can you draw the graph G = (V, A) and write down its weight matrix W
where V = {a, b, c, d, e, f}, the arcs A = {ab, bc, be, cf, ef, ad, de} and the
corresponding weights are [2, 1, 4, 5, 1, 3, 2].
5. Practical Project: [use Chapter 5: Graph Theory].
A farmer wishes to ferry a deer, a cabbage, a lion and a pet mouse across
a river with a small boat which can accommodate at most two of his
belongings at a time. Furthermore, he
i/ will not let the lion be alone (i.e. without him) with the deer, and also
ii/ will not let the mouse be alone with the cabbage,
for a clear reason that the predators will eat the preys in the three
mentioned cases.
a/ Model the problem with a state transition graph G = (V, E) by using
appropriate state variables; f for the farmer, and g, c, w, m for his
belongings. Known that state variables can receive two states: L (for
at the left bank) and R (for at the right bank).
7.4. PROBLEMS 113
Hint: Describe firstly all states (vertices), and then list eligible states
u of V , by using the system invariants mentioned above with logic
notation.
b/ Describe clearly rules to form edges of G.
Appendix 1: Hamming metric.
Usually a state set of a graph G = (V, E) can be given by a binary
space/set V consisting of length n vectors:
V = {u = (u
1
, u
2
, . . . , u
n
) : u
i
= 0 u
i
= 1}.
The Hamming distance d(u, v) between two binary states
u = (u
1
, u
2
, . . . , u
n
) and v = (v
1
, v
2
, . . . , v
n
) is the number of their distinct
coordinates:
d(u, v) = |u
1
v
1
| +|u
2
v
2
| +. . . +|u
n
v
n
| =
n

i=1
|u
i
v
i
|
The weight of a binary state/vector is defined to be
wt(u) = d(u, 0) =
n

i=1
|u
i
0| =
n

i=1
u
i
.
The Hamming distance d(., .) defined on some binary space V is also
called the Hamming metric, and the space V equipped with the Hamming
metric d(., .) is called a Hamming metric space.
Appendix 2: State-transition in graphs.
Very often, changing states of a state-transition graph G = (V, E) can be
conducted mathematically by measuring how far the Hamming distance is
114 CHNG 7. PROBABILISTIC MODELS
between an original state u = (u
1
, u
2
, . . . , u
n
) to its effect state
v = (v
1
, v
2
, . . . , v
n
).
Example 18. A farmer wishes to ferry a goat, a cabbage and a wolf
across a river with a small boat which can
accommodate at most one of his belongings at a time.
Furthermore, he will not let the goat be alone (i.e. without him) with
cabbage, for a clear reason that the predator will eat the prey in the
mentioned case. What would be possible invariants of the process of
transporting between the river banks?
* You can not formulate any invariant without introducing some major
variables. Here four variables f, g, c, w would be reasonably good for
describing the moving process, where f, g, c, w all receive values from the
binary set {L(eft bank), R(ight bank)} = {0, 1}. Well, the situation that
the goat is not allowed be alone with cabbage without farmer is just
symbolically represented by the fixed rule:
Invariant 1 : (g = c) OR (f = g = c)
This rule is indeed an invariant, since it is (or better must be) maintained
in the whole process.
* Finding the invariant saying that the small boat can
accommodate at most one of the farmer belongings at a time is more
tricky! To do this, you have to think about the state-changing aspect of
the process when the farmer rows from one side to the other side of the
river. [You will not draw a boat with some items on it and move it on the
river several times, will you?] Then a simple question naturally arises:
7.4. PROBLEMS 115
what are the states of the process?
Example 19 (The farmers crossing river problem, cont.). The states of
the river crossing process are binary vectors of length 4
u = (u
f
, u
g
, u
c
, u
w
) = (u
1
, u
2
, u
3
, u
4
) {0, 1}
4
,
if we encode the left bank L and the right bank R by {0, 1} as done above!
We understand when the farmer is rowing his boat, for instance from a
left river bank u = (u
1
, u
2
, u
3
, u
4
) to a right river bank v = (v
1
, v
2
, . . . , v
n
)
(or the other way round), his position must change. The changing of state
u to v creates an edge e = (u, v) E, indeed! More clearly, the edge
e = (u, v) is truly determined iff
if u
1
= L(i.e. 0) then v
1
= R(i.e. 1); or the other way round.
A ha, we just find another invariant, must always be true during process
evolving or running: [e = (u, v) would exist if we have ]
Invariant 2 : u
1
+v
1
= 1, where the sum is binary plus.
Can you find more invariants and employ them?
.....
ANSWER: Combining with the fact that
the small boat can accommodate at most one of the farmer belongings
we realize a starting state u change at most two of its two coordinates to
be the result v. Hence, the third invariant is found:
116 CHNG 7. PROBABILISTIC MODELS
Invariant 3 : d(u, v) =
4

i=1
|u
i
v
i
| 2.
7.4. PROBLEMS 117
Copyright in 2011 by
Lecturer Nguyen V.M. Man, Ph.D.
Faculty Computer Science and Engineering
Institution University of Technology of HCMC
Address 268 Ly Thuong Kiet, Dist. 10, HCMC, Vietnam
Email: [email protected]
[email protected]