Study Guide for Exam 1, Linear
Optimization
Mariano Echeverria
Topics covered:
ë The exam will be based on the material from Lectures 1,2,3,4,5,6.
ë If you are reading the book, sections 1.1, 1.3, 2.1, 2.2, 2.5, 4.1, 4.2, 4.3 (the other
sections from those chapter provide interesting information which can be read for
those who want to dig further).
1 Some conversion rules:
ë Any inequality ≥ can be turned into ≤ by multiplying by −1 and reversing the
inequality. Example:
4 ≤ x − y + 3z ⇐⇒ −x + y − 3z ≤ −4
ë Likewise, an equality can be converted into two inequalities. For example,
2x + y + z = 5 ⇐⇒ 2x + y + z ≤ 5 and 5 ≤ 2x + y + z
ë An inequality ≤ can be turned into an equality by adding a slack variable s. For
example,
x + 2y + z ≤ 3 ⇐⇒ x + 2y + z + s = 3
where s ≥ 0 is the slack variable.
ë You may check problems Converting to Standard Form 1 , Converting to Standard
Form II from the problems pdf.
2 Setting up Linear Optimization Problems
ë Here you are given a word problem, and you must rewrite in mathematical form.
The problems Lincoln Outdoors, Transportation Problem, Separation of Points,
Diet Problem, Selling Coffee are a good sample of what I have in mind.
1
ë If needed, you will need to rewrite it in the form
max(c · x) or min(c · x)
x≥0
Ax ≤ b
where c = (c1 , c2 , · · · , cn )T , x = (x1 , x2 , · · · , xn )T , b = (b1 , b2 , · · · , bm )T (here T
represents the transpose so that one regards the vector as a column vector), and A
is an m × n matrix.
ë For these problems I am more interested in setting them up, so I won’t ask you to
solve the optimization problem.
3 Convexity
ë A convex region must contain all line segments connecting any two points within
the region.
ë Mathematically, to check if a region K is convex, you need to check that for any
0 ≤ λ ≤ 1, and for any two points x1 , x2 within K, the point (1 − λ)x1 + λx2 is
also within K.
ë For concrete regions (like those from the problem Convexity Examples from the
problems pdf), a proof by picture is ok to justify that a region is convex. Likewise,
a picture counterexample is enough to justify why a region wouldn’t be convex.
ë However, there are proof-type problems about convexity I would expect you to
know. The problems Intersection and Union of Convex Sets, Convexity and Ho-
mogeneous System are good examples of this. It is also important to understand
the proof we did in class that the region K = {x ∈ Rn which solve Ax = b} is
convex.
4 Basic Solutions
ë After introducing slack variables if needed, we focus here in problems of the form
max(c · x) or min(c · x)
x≥0
Ax = b
ë Some Terminology: if a vector x solves Ax = b and in addition satisfies x ≥ 0,
then x is called a feasible solution.
ë Basic assumption: our matrix A is an m × n matrix with rank(A) = m and
m < n (so there are more columns than rows for A).
ë More terminology: A basis for the matrix A is a selection of m columns of
the matrix A with the property that the sub-matrix formed by these columns is
2
invertible. For example, for the matrix
1 0 2 2
A=
1 1 3 2
selecting columns 1 and 3 yields a basis since
1 2
A{1,3} =
1 3
is invertible (because it has non-zero determinant). However, choosing columns 1
and 4 does not yield a basis since
1 2
A{1,4} =
1 2
is not invertible (has zero determinant).
ë A basis for A can be used to produce a solution to the system Ax = b which is
called a basic solution. If you call B the columns that were selected, and AB the
corresponding square matrix, then the basic solution is
xB = (A−1
B b, 0)
however, the previous formula needs to be interpreted carefully, since it doesn’t
mean that the last entries of the vector is full of 0s. It just means that we set equal
to 0 the variables which were not chosen by the basis B.
ë For example, for the matrix A we were studying, suppose that we are solving
x1
1 0 2 2 x2 2
=
1 1 3 2 x3 4
x4
1 2
ë If we choose the basis B = {1, 3}, we had AB = , and thus we would
1 3
solve
1 2 x1 2
=
1 3 x3 4
| {z } | {z } | {z }
AB xB b
which has solution
x1 −1 3 −2 2 −2
= AB b = =
x3 −1 1 4 2
and then basic the solution is obtained from xB by making the remaining variables
3
(called non-basic variables) equal to 0. Thus
−2
x1
x 0
x= 2 =
x3 2
x4 0
Notice that x is a basic solution but it is not feasible since it doesn’t satisfy the
requirement that all its entries are non-negative.
1 0
ë On the other hand, consider the basis B = {1, 2}, we have AB = , and
1 1
thus we would solve
1 0 x1 2
=
1 1 x2 4
| {z } | {z } | {z }
AB xB b
which has solution
x1 1 0 2 2
= A−1
B b = =
x2 −1 1 4 2
Thus
x1 2
x 0
x= 2 =
x3 2
x4 0
this solution is basic and feasible, since all its entries are non-negative.
ë Notice that for a basic solution xB , all the variables which do not correspond to
the columns that were chosen for the basis are set equal to 0. This does not mean
that all the variables which appear in xB must be non-zero. In fact, if one of the
basic variables which appear in xB happens to be 0, then the solution x is called a
degenerate solution.
ë For example, consider the (very simple) optimization problem
max(x2 )
x1 ≥ 0, x2 ≥ 0
−x + x2 ≤ 0
1
x1 ≤ 2
After introducing slack variables we end up considering
max(x2 )
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0, x4 ≥ 0
−x + x2 + x3 = 0
1
x1 + x4 = 2
4
which has the matrix representation
x1
−1 1 1 0 x2 0
1 0 0 1 x = 2
3
x4
Consider the basis B = {2, 4}, then
1 0
AB =
0 1
and in this case
x2 0 0
= A−1
B =
x4 2 2
x1 0
x2 0
since the basic variable x2 is 0, then the basic solution = is a
x3 0
x4 2
degenerate basic solution.
ë For this example, notice that if we had chosen the basis B = {3, 4}
1 0
AB =
0 1
and in this case
x3 0 0
= A−1
B =
x4 2 2
x1 0
x2 0
since the basic variable x3 is 0, then the basic solution = , but
x3 0
x4 2
this is the same basic solution as the one we obtained from the basis B = {2, 4}!!!
In fact, another way to characterize a degenerate basic solution is that
there are distinct basis for the matrix A which produce it.
ë For this topic study problems, Finding Basic Solutions I, Finding Basic solutions
II, Finding Basic solutions III, Finding basic solutions IV.
