Compulsory Project 2 – with answers

MAT-INF3100 Linear optimization, Spring 2015

• NOTE. Hand in your assignment using Devilry (devilry.ifi.uio.no) in a

single PDF file that you name “username.pdf” (your username!). You
must also hand in the source code you wrote for Problem 3 (You are ex-
pected to discuss it as well as hand it in). Moreover, you should read the
general information about compulsory projects at the course web page.
Your submission should be written by yourself and reflect your own un-
derstanding.

Problem 1
1.1
Write down the dual problems for the following LP problems:

min 3x1 + 5x2 − x3 ,

subject to
x1 − x2 + x3 ≤ 3, (1)
2x1 − 3x2 ≤ 4,
x1 , x2 ≥ 0.

and
max 3x1 + 2x2 ,
subject to
4x1 + 2x2 ≤ 16,
(2)
x1 + 2x2 ≤ 8,
x1 + x2 ≤ 5,
x1 , x2 ≥ 0.

Answer: For (1), the dual problem reads

max −3y1 − 4y2 ,

subject to
−y1 − 2y2 ≤ 3,
(3)
y1 + 3y2 ≤ 5,
−y1 ≤ −1,
y1 , y2 ≥ 0.

1
Answer: For (2), the dual problem reads
min 16y1 + 8y2 + 5y3 ,
subject to
4y1 + y2 + y3 ≥ 3, (4)
2y1 + 2y2 + y3 ≥ 2,
y1 , y2 ≥ 0.

1.2
Write (1) and its dual problem in matrix form. Do the same for (2).

Answer: The primal problem (1) can be written as

min c> x, Ax ≥ b, x ≥ 0,
where x = (x1 , x2 , x3 ) , c = (3, 5, −1) , b = (3, 4)> , and
> >
 
1 −1 1
A= .
2 −3 0
The dual problem can be written as
max b> y, A> y ≤ c, y ≤ 0,
where y = (y1 , y2 )> .
Similarly, the primal problem (2) can be written as
max c> x, Ax ≥ b, x ≥ 0,
where x = (x1 , x2 ) , c = (3, 2) , b = (16, 8, 5)> , and
> >
 
4 2
A = 1 2 .
1 1
The dual problem can be written as
max b> y, A> y ≤ c, y ≤ 0,
where y = (y1 , y2 , y3 )> .

1.3
Show that x? = (3, 2) is feasible for the primal problem (2) and y ? = (1/2, 0, 1)
is feasible for the corresponding dual problem. Moreover, show that x? is in fact
the optimal solution of (2).

Answer:: Straightforward computations reveal that x? = (3, 2) and y ? = (1/2, 0, 1)

satisfy their respective constraints. Moreover, the objective function values of
the primal and dual problems coincide:
η ? := 3 ∗ 3 + 2 ∗ 2 = 9 + 4 = 13, ξ ? := 16 ∗ 1/2 + 8 ∗ 0 + 5 ∗ 1 = 13.
We then conclude by the weak duality theorem.

2
Problem 2
Consider the function
S(x, y) = f (x) − xy + g(y), (x, y) ∈ R2 , R = (−∞, ∞),
where f is a strongly concave function and g is a strongly convex function.
A function h is called strongly convex if h00 (y) > 0 for all y ∈ R, h0 (y) → −∞
as y → −∞, and h0 (y) → ∞ as y → ∞. A function is strongly concave if −h is
strongly convex.

2.1
Show that the gradient ∇S = (∂S/∂x, ∂S/∂y) vanishes at a unique point
(x? , y ? ) ∈ R2 .

Answer: Computing partial derivatives,

∂S/∂x = f 0 (x) − y = 0, ∂S/∂y = g 0 (y) − x = 0,
from which it follows that
φ(y) = y, ψ(x) = x,
where
φ(y) = f 0 (g 0 (y)), ψ(x) = g 0 (f 0 (x)).
These functions are decreasing and maps R onto R:
φ0 (y) = f 00 (g 0 (y))g 00 (y) < 0, ψ 0 (x) = g 00 (f 0 (x))f 00 (x) < 0,
as f is concave and g is convex. Hence, the graph of φ(y) must intersect the
straight line y at a unique point y ∗ . Similarly, ψ(x) must intersect the straight
line x at a unique point x∗ .

2.2
With the stationary point (x? , y ? ) ∈ R2 given above, show that
max min S(x, y) = S(x? , y ? ), min max S(x, y) = S(x? , y ? ).
x∈R y∈R y∈R x∈R

Hint: Verify second order derivative tests for both inner and outer optimisations.

Answer: Let us first consider maxx∈R miny∈R S(x, y). We compute

∂ 2 S/∂y 2 = g 00 (y) > 0 (as g is convex).
Hence, miny∈R S(x, y) = S(x, y ? ), for each fixed x. Next, we compute
∂ 2 S/∂x2 = f 00 (x) < 0 (as f is concave),
and so maxx∈R S(x, y ? ) = S(x? , y ? ). Hence, maxx∈R miny∈R S(x, y) = S(x? , y ? ).
A similar argument gives that miny∈R maxx∈R S(x, y) = S(x? , y ? ).

3
Problem 3
Assume we have an LP problem on standard form:

max {cT x : Ax ≤ b, x ≥ O}

For simplicity we only work with matrices A ≥ 0 and assume that b ≥ 0, so

x = 0 is a feasible solution (A is an m times n matrix, and b, c are appropriately
sized). Given this, implement the following two algorithms:

• Algorithm S: the primal simplex algorithm (Section 6.2 in the book)

• Algorithm I: the path-following interior point algorithm (Figure 18.1 in
the book, see chapters 17, 18 and 19 for various details)

The choice of language is up to you, but we recommend MATLAB (or per-

haps Python). Hand in your code (in separate files) and a test run for both
algorithms, using the data file on the course web page as a test. Print a line or
two of useful information every iteration. Discuss your results.
Some further points:

• The optimal value in the example problem is 212.6539.

• When solving linear systems of equations V x = d do not use the inv(V)
command – rather use the backslash operator, i. e. x = V \ d.
• The major part of the work for algorithm I is solving the KKT-system
(Karush-Kuhn-Tucker system, the optimality conditions for the barrier
problem) to find the steps ∆x, ∆y, ∆z and ∆w. This can be done in
different ways, but we suggest using the normal equations in primal form,
see start of section 19.2, eq. (19.9) for finding ∆y, then find ∆x etc.
• If you are using matlab and have downloaded the .mat file from the web-
page, you can use the command load(’lptestproblemoblig.mat’) to
have the correct A, b, c appear in your MATLAB workspace.
• The test problem is quite large, so it might be a good idea to develop
against a simpler problem at first – see for example pages 96-101 in Van-
derbei (2nd ed.).

Answer: Note: We do not include the code in this file – in a proper solution it
would be attached as two separate files, as described in the assignment.
The following is an example of output when running the two algorithms.
Running s i m p l e x method . . .
I t e r a t i o n 1 complete , c u r r e n t value 108.0549
I t e r a t i o n 2 complete , c u r r e n t value 139.9851
I t e r a t i o n 3 complete , c u r r e n t value 160.3693
I t e r a t i o n 4 complete , c u r r e n t value 163.0008
I t e r a t i o n 5 complete , c u r r e n t value 164.7146
I t e r a t i o n 6 complete , c u r r e n t value 172.6429

4
Iteration 7 complete , current value 186.0797
Iteration 8 complete , current value 192.3676
Iteration 9 complete , current value 196.2962
Iteration 10 complete , current value 201.5087
Iteration 11 complete , current value 202.8033
Iteration 12 complete , current value 203.2862
Iteration 13 complete , current value 203.6421
Iteration 14 complete , current value 203.7756
Iteration 15 complete , current value 208.0495
Iteration 16 complete , current value 209.4698
Iteration 17 complete , current value 209.4733
Iteration 18 complete , current value 209.6871
Iteration 19 complete , current value 211.3956
Iteration 20 complete , current value 212.0439
Iteration 21 complete , current value 212.0981
Iteration 22 complete , current value 212.1842
Iteration 23 complete , current value 212.5634
Iteration 24 complete , current value 212.5669
Iteration 25 complete , current value 212.6539

Running i n t e r i o r p o i n t method . . .
I t e r a t i o n 1 complete , c u r r e n t c o m p l e m e n t a r i t y 1 1 4 . 2 9 4 4
I t e r a t i o n 2 complete , c u r r e n t c o m p l e m e n t a r i t y 8 5 . 5 5 4 1
I t e r a t i o n 3 complete , c u r r e n t c o m p l e m e n t a r i t y 6 3 . 6 6 8 8
I t e r a t i o n 4 complete , c u r r e n t c o m p l e m e n t a r i t y 4 4 . 0 3 3 2
I t e r a t i o n 5 complete , c u r r e n t c o m p l e m e n t a r i t y 3 4 . 9 9 2 2
I t e r a t i o n 6 complete , c u r r e n t c o m p l e m e n t a r i t y 2 9 . 1 3 1 9
I t e r a t i o n 7 complete , c u r r e n t c o m p l e m e n t a r i t y 2 5 . 7 2 2 4
I t e r a t i o n 8 complete , c u r r e n t c o m p l e m e n t a r i t y 2 2 . 6 4 7 3
I t e r a t i o n 9 complete , c u r r e n t c o m p l e m e n t a r i t y 1 7 . 3 6
I t e r a t i o n 10 complete , c u r r e n t c o m p l e m e n t a r i t y 1 4 . 0 1 9 8
I t e r a t i o n 11 complete , c u r r e n t c o m p l e m e n t a r i t y 1 1 . 1 1 3 1
I t e r a t i o n 12 complete , c u r r e n t c o m p l e m e n t a r i t y 1 0 . 3 2 3 8
I t e r a t i o n 13 complete , c u r r e n t c o m p l e m e n t a r i t y 8 . 2 4 2 1
I t e r a t i o n 14 complete , c u r r e n t c o m p l e m e n t a r i t y 5 . 8 7 8 2
I t e r a t i o n 15 complete , c u r r e n t c o m p l e m e n t a r i t y 4 . 0 6 3 4
I t e r a t i o n 16 complete , c u r r e n t c o m p l e m e n t a r i t y 2 . 8 1 3 3
I t e r a t i o n 17 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 9 1 4 7
I t e r a t i o n 18 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 1 8 5 4
I t e r a t i o n 19 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 7 7 2 9 4
I t e r a t i o n 20 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 3 9 5 2 6
I t e r a t i o n 21 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 1 8 3 2 2
I t e r a t i o n 22 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 8 5 9 4 5
I t e r a t i o n 23 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 3 5 1 5 3
I t e r a t i o n 24 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 9 7 9 8 5
I t e r a t i o n 25 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 1 3 5 2 8
I t e r a t i o n 26 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 0 1 7 3 2
I t e r a t i o n 27 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 9 8 9 6 e −05
I t e r a t i o n 28 complete , c u r r e n t c o m p l e m e n t a r i t y 2 . 0 6 0 6 e −06

5
Runs c o m p l e t e . Values :
SIMPLEX : 2 1 2 . 6 5 3 9
INTERIOR : 2 1 2 . 6 5 3 9
Max . d e v i a t i o n : 9 . 4 8 4 4 e −07
In terms of iteration count, the two methods were comparable here, but
we know from the theory that for larger problems this would not be the case.
We also see that the two found solutions are more or less exactly the same, as
the maximal deviation between the two is of order 10−6 – this could indicate
that the interior point algorithm converges to the same solution as the simplex
method in this case.