Cutting Plane Method

Lecture 5
Cutting Plane Method Introduction

Method of Integer
Forms for IP
MATH3220 Operations Research and Logistics Mixed cut for MILP
Jan. 20, 2015 Row generation for the
TSP

Pan Li
The Chinese University of Hong Kong
5.1
 Cutting Plane Method
Agenda

Introduction
1 Introduction Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP
2 Method of Integer Forms for IP

3 Mixed cut for MILP

4 Row generation for the TSP

5.2
 Cutting Plane Method

The basic idea of the cutting plane method is to cut off

parts of the feasible region of the LP relaxation, so that the
optimal integer solution becomes an extreme point and
therefore can be found by the simplex method.
Introduction
The cutting plane algorithm in its general form can then be Method of Integer
Forms for IP
formulated as follows:
Mixed cut for MILP
Step 1: Solve the LP relaxation. Get x ∗ . Row generation for the
Step 2: If x ∗ integral stop, else find a valid inequality that will TSP

exclude x ∗ .
Step 3: Go to step 1.

In practice, the branch-and-bound method almost

outperform the cutting plane method.

Cutting plane is the first algorithm developed for integer

programming that could be proved to converge in a finite
number of steps.

Even though the algorithm is considered not efficient, it

has provided insights into integer programming that have
led to other, more efficient, algorithms.
5.3
 Cutting Plane Method
Example

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.4
 Cutting Plane Method
Method of Integer Forms for IP

We assume all problems are comprised of rational data (notice

that all numbers stored in digital computers are rational) and
since rational data can be transformed into integral data by
Introduction
multiplying by a suitable constant, we shall only consider Method of Integer
problems comprised of integral data. Forms for IP

Mixed cut for MILP

Let an ILP be given. Assume that a non-integer optimal Row generation for the

solution x∗ of the corresponding LP has been found, and that TSP

the optimal tableau is written as

· · · xr · · · · · · xi · · · solution
xr I · · · yri · · · xr∗
z 0 z∗

where {xr : r ∈ J} are the basic variables and {xi : i ∈

/ J} are
the non-basic variables.
Let us denote the set of feasible integer solutions by FI .

5.5
 Cutting Plane Method
Suppose that x∗ has a non-integer component xr∗ . Then for
any x ∈ FI , from the r th constraint, we have
X
xr + yri xi = xr∗ . (1)
i ∈J
/
Introduction
Since x ≥ 0, X Method of Integer
xr + byri cxi ≤ xr∗ . (2) Forms for IP

Mixed cut for MILP

i ∈J
/
Row generation for the
Since for x ∈ FI , it has integer components, the L.H.S. of (2) TSP

must be an integer. Thus

X
xr + byri cxi ≤ bxr∗ c. (3)
i ∈J
/

Subtracting (3) from (1), we get

X
(yri − byri c)xi ≥ xr∗ − bxr∗ c. (4)
i ∈J
/

Letting yri = byri c + fri and xr∗ = bxr∗ c + fr , we obtain (a Gomory

cut) X
fri xi ≥ fr , ∀ x ∈ FI . (5)
i ∈J
/
Hence (5) does not exclude any x ∈ FI . 5.6
 Cutting Plane Method

The current optimal solution will be excluded by (5)

X
fri xi ≥ fr , ∀ x ∈ FI . (5)
i ∈J
/
Introduction
xr∗ − bxr∗ c = fr > 0. Method of Integer
Forms for IP

⇒ the right hand side of (5) is positive. Mixed cut for MILP

Row generation for the

∗
However, at the current optimal solution x , xi = 0 for all i 6∈ J, TSP

and hence
X
fri xi = 0 ∀ i∈ /J
i ∈J
/

Thus the current optimal solution is excluded by (5) as it does

not satisfy (5).

5.7
 Cutting Plane Method

Define the surplus variable of (5) as s:

X
s := −fr + fri xi with s ≥ 0. (6)
i ∈J
/
Introduction
We note that s must be integral for all x ∈ FI since by (1), Method of Integer
Forms for IP
X X X Mixed cut for MILP
xr = −(−fr + fri xi )+([xr∗ ]− [yri ]xi ) = −s+([xr∗ ]− [yri ]xi ) Row generation for the
i ∈J
/ i ∈J
/ i ∈J
/ TSP

and both xr and the expression inside the second parenthesis

on the R.H.S. are integral.

5.8
 Cutting Plane Method
The Gomory Method

How to use valid inequality? Consider the integer program

max{cx : Ax ≤ b, x ≥ 0 and integer } Introduction

Method of Integer
Forms for IP
Idea: Mixed cut for MILP

Row generation for the

First solve the associated LP relaxation and then generate a TSP

C-G inequality on the constraint associated with the basic

variable that is not integer (if there is any), resolve the new LP
relaxation...

5.9
 Cutting Plane Method
Example

Given a problem in standard form:

min z = −2x1 − x2
 Introduction
 x1 +x2 +x3 = 5,
Method of Integer

−x1 +x2 +x4 = 0,

Forms for IP
subject to

 6x1 +2x2 +x5 = 21, Mixed cut for MILP

x1 , x2 , ..., x5 ≥ 0 and integer.

 Row generation for the
TSP

Solve the LP which yields the following optimal tableau:

x1 x2 x3 x4 x5 solution
x1 1 0 − 12 0 1
4
11
4
3
x2 0 1 2 0 − 14 9
4
1 1
x4 0 0 −2 1 2 2

z 0 0 − 12 0 − 14 − 31
4

Note that it is optimal because all the reduced cost coefficients

are nonpositive (recall that it is a minimization problem).
5.10
 Cutting Plane Method
Example

x1 x2 x3 x4 x5 solution
x1 1 0 − 12 0 1
4
11
4
3
x2 0 1 2 0 − 14 9
4
Introduction

Method of Integer
1 1 Forms for IP
x4 0 0 −2 1 2 2 Mixed cut for MILP

z 0 0 − 12 0 − 14 − 31
4
Row generation for the
TSP

Each row could serve as the source of a cut, or simply as a

sourceProw.
/ fri xi ≥ fr to generate a cut yields
Using i ∈J

x3 /2 + x5 /4 ≥ 3/4 (x1 row),

x3 /2 + 3x5 /4 ≥ 1/4 (x2 row),
x5 /2 ≥ 1/2 (x4 row). (7)

5.11
 Cutting Plane Method
Example
Suppose we add the x4 row cut, or

−x5 /2 + s1 = −1/2, s1 ≥ 0
Introduction
First, from the tableau, we have Method of Integer
Forms for IP

1 1 Mixed cut for MILP

−2x3 + x4 + x5 = . (8) Row generation for the
2 2 TSP

This is to be satisfied by all x ∈ F, and in particular by all

x ∈ FI .
∵ x5 ≥ 0, ⇒ −2x3 + x4 ≤ 21 ⇒ −2x3 + x4 ≤ 0.
Subtracting that from (8), we have the cutting-plane

x5 /2 ≥ 1/2 (7)
Note that since we derive (7) by assuming that x ∈ FI , we do
not exclude any x ∈ FI .
Moreover, since at the current optimal solution, we have x5 = 0.
Hence x5 /2 6≥ 1/2. That means the current optimal solution will
be excluded by the cutting-plane.
5.12
 Cutting Plane Method
Example

Gomory cut in terms of x1 and x2 :

x5 ≥ 1 ⇒ (21 − 6x1 − 2x2 ) ≥ 1 ⇒ 10 ≥ 3x1 + x2
Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.13
 Cutting Plane Method
Example

Add the cutting plane, yields the next table with s1 = −1/2 as a
new basic variable.
x1 x2 x3 x4 x5 s1 solution Introduction

x1 1 0 − 12 0 1
4 0 11
4
Method of Integer
Forms for IP

3
− 14 9 Mixed cut for MILP
x2 0 1 2 0 0 4 Row generation for the
1 1 TSP
source x4 0 0 −2 1 2 0 2
cut s1 0 0 0 0 − 21 1 − 12
z 0 0 − 12 0 − 14 0 − 31
4

Dual feasibility buy not primal feasibility is retained.

⇒ Reoptimize using Dual simplex method.
⇒ s1 leaves and x5 enters the basis.

5.14
 Cutting Plane Method
Example

After pivoting, we obtain the next table (excluding the s2 row

and column):
x1 x2 x3 x4 x5 s1 s2 solution Introduction

Method of Integer
source x1 1 0 − 12 0 0 1
2 0 10
4
Forms for IP

Mixed cut for MILP

3
x2 0 1 2 0 0 − 12 0 10
4 Row generation for the
TSP
x4 0 0 −2 1 0 1 0 0
x5 0 0 0 0 1 −2 0 1
cut s2 0 0 − 12 0 0 − 12 1 − 12
z 0 0 − 12 0 0 − 12 0 − 30
4

xr∗ ≥ 0, ⇒ Primal feasible.

Reduced cost coefficients ≤ 0, ⇒ tableau is optimal.
The current optimal solution is
(x1 , x2 , x3 , x4 , x5 , s1 ) = (10/4, 10/4, 0, 0, 1, 0).

5.15
 Cutting Plane Method
Example

x1 x2 x3 x4 x5 s1 s2 solution
source x1 1 0 − 12 0 0 1
2 0 10
4
Introduction
3
x2 0 1 2 0 0 − 12 0 10
4 Method of Integer
Forms for IP
x4 0 0 −2 1 0 1 0 0 Mixed cut for MILP

Row generation for the

x5 0 0 0 0 1 −2 0 1 TSP

cut s2 0 0 − 12 0 0 − 12 1 − 12
z 0 0 − 12 0 0 − 12 0 − 30
4

We can choose x1 or x2 row as a source row. Arbitrarily

selecting the x1 row, we obtain

x3 /2 + s1 /2 ≥ 1/2.
Thus we add the constraint

−x3 /2 − s1 /2 + s2 = −1/2

to the table above and reoptimize. 5.16

 Cutting Plane Method
Example

After pivoting, we obtain

x1 x2 x3 x4 x5 s1 s2 solution
Introduction
source x1 1 0 0 0 0 1 1 3
Method of Integer
Forms for IP
x2 0 1 0 0 0 −2 3 1 Mixed cut for MILP

x4 0 0 0 1 0 3 −4 2 Row generation for the

TSP

x5 0 0 0 0 1 −1 0 1
cut x3 0 0 1 0 0 1 −2 1
z 0 0 0 0 0 0 −1 −7

The solution is primal feasible, optimal and integral; therefore it

is optimal to the IP.

5.17
 Cutting Plane Method
Example

Gomory cut in terms of x1 and x2 :

x3 + s1 ≥ 1 ⇒ 2x3 + (x5 − 1) ≥ 2
⇒ 2(5 − x1 − x2 ) + (21 − 6x1 − 2x2 − 1) ≥ 2 ⇒ 7 ≥ 2x1 + x2 .
Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.18
 Cutting Plane Method
Mixed cut for MILP

Let xr be an integer variable of the MILP.

Consider the xr -equation in the optimal continuous solution.
This is given by
Introduction
X
xr + yri xi = xr∗ (Source row) Method of Integer
Forms for IP
i Mixed cut for MILP

and X Row generation for the

TSP
xr + yri xi = bxr∗ c + fr (9)
i
Because some of the nonbasic variables may not be restricted
to integer values in this case, it is incorrect to use the cut
developed in the preceding section.
For xr to be integer, either xr ≤ bxr∗ c or xr ≥ bxr∗ c + 1 must be
satisfied. From the source row, these conditions are equivalent
to:
X
yri xi ≥ fr (10)
i
X
yri xi ≤ fr − 1 (11)
i
5.19
 Cutting Plane Method

Let
J + = set of subscripts of non-basic variables for which yri ≥ 0
J − = set of subscripts of non-basic variables for which yri < 0
Then from the above inequalities we obtain the following two
inequalities Introduction
X Method of Integer
yri xi ≥ fr (12) Forms for IP

Mixed cut for MILP

i∈J +
 X Row generation for the
fr TSP
yri xi ≥ fr (13)
fr − 1
i∈J −

One of these two inequalities must hold, hence we can

combine them to give the inequality:
 X
X fr
yri xi + yri xi ≥ fr
+
fr − 1 −
i∈J i∈J

By adding one slack variable s, it becomes the "mixed cut

equation":
 X !
X fr
s− yri xi + yri xi = −fr
+
fr − 1 −
i∈J i∈J
5.20
 Cutting Plane Method
Example

max z
= 2x1 + x2

 x1 +x2 +x3 = 5, Introduction

−x +x2 +x4 = 0, Method of Integer


1

 Forms for IP
subject to 6x1 +2x2 +x5 = 21, Mixed cut for MILP

x1 , x2 , ..., x5 ≥ 0


 Row generation for the
 TSP
x1 integer


Solve the LP which yields the following optimal tableau:

x1 x2 x3 x4 x5 solution
x1 1 0 − 12 0 1
4
11
4
3 1 9
x2 0 1 2 0 −4 4
1 1
x4 0 0 −2 1 2 2
1 1 31
z 0 0 2 0 4 4

5.21
 Cutting Plane Method
Example

x1 x2 x3 x4 x5 solution
x1 1 0 − 12 0 1
4
11
4
3
x2 0 1 2 0 − 14 9
4
Introduction

Method of Integer
1 1 Forms for IP
x4 0 0 −2 1 2 2 Mixed cut for MILP
1 1 31
z 0 0 2 0 4 4
Row generation for the
TSP

From the x1 -row in tableau, the mixed cut is

( !  )
3
1 4 1 3
s1 − x5 + 3 − x3 = − ,
4 4 − 1 2 4
or
1 3 3
s1 − x5 − x3 = −
4 2 4

5.22
 Cutting Plane Method
Example

x1 x2 x3 x4 x5 s1 solution
x1 1 0 − 12 0 1
4 0 11
4
3
x2 0 1 2 0 − 14 0 9
4 Introduction

1 1 Method of Integer
source x4 0 0 −2 1 2 0 2
Forms for IP

Mixed cut for MILP

cut s1 0 0 − 32 0 − 14 1 − 34 Row generation for the
TSP
1 1 31
z 0 0 2 0 4 0 4

Applying the dual simplex method we obtain finally the tableau.

x1 x2 x3 x4 x5 s1 solution
1
x1 1 0 0 0 3 − 13 3
1 3
x2 0 1 0 0 −2 1 2
5
source x4 0 0 0 1 6 − 43 3
2
1
cut x3 0 0 1 0 6 − 23 1
2
1 1 15
z 0 0 0 0 6 3 2

⇒ Optimal solution x1 = 3, x2 = 3/2 and the maximum value of

the objective function is 15/2. 5.23
 Cutting Plane Method
Example
Question: Can you write down the "geometric equation" of the
MILP cut?
Doing this, you can see better why the cut only cut out x1 and
does not affect x2 . Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

max z= 2x1 + x2
Row generation for the

 x1 +x2 +x3 = 5, TSP

−x1 +x2 +x4 = 0,




subject to 6x1 +2x2 +x5 = 21,
x1 , x2 , ..., x5 ≥ 0




x1 integer


1 3 3
s1 − x5 − x3 = −
4 2 4

5.24
 Cutting Plane Method
Example

The cut is
3 1 3
x3 + x5 ≥
2 4 4 Introduction
3 1 3 Method of Integer
⇒ (5 − x1 − x2 ) + (21 − 6x1 − 2x2 ) ≥ Forms for IP
2 4 4
Mixed cut for MILP
⇒3x1 + 2x2 ≤ 12 Row generation for the
TSP

5.25
 Cutting Plane Method
Disadvantages of Cutting plane method

(i) When this algorithm is implemented on digital computer,

round-off errors may cause serious problem, especially in
distinguishing integers from non-integers.
Introduction

Method of Integer
(ii) The solution of the problem remains infeasible until the Forms for IP

optimal integer solution is reached. This means that there Mixed cut for MILP

Row generation for the

will be no “good” integer solution on hand, if the TSP

calculations are stopped prematurely owing to budget or

time limitations.

5.26
 Cutting Plane Method
Cutting Plane algorithm

Overall strategy:
1 Solve LP relaxation, get relaxed solution x ∗ .
2 If x ∗ is a feasible solution of P, exit. Introduction

∗ Method of Integer
3 Use solution x of the relaxed one to construct a valid cut C Forms for IP

for P. Mixed cut for MILP

Row generation for the

4 Add the constraint C to the formulation of P. TSP

5 Go back to 1.

Crux of the algorithm: step 3 (separation problem)

Cutting plane algorithms may depends on the particular

problem structure of be completely general.

Independent of problem structure: Gomory cutting planes

Problem structure: Row generation for the TSP

5.27
 Cutting Plane Method
Travelling Salesman Problem

Given a collection of cities and the cost of travel between each

pair of them, the travelling salesman problem, or TSP for short,
is to find the cheapest way of visiting all the cities and returning
Introduction
to the starting point. Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.28
 Cutting Plane Method
Comments on TSP

First formulated as a mathematical problem in 1930 and is

one of the most intensively studied problems in
optimization.
Introduction

Method of Integer
Applications: planning, logistics, microchip design, DNA Forms for IP

sequencing, and more. Mixed cut for MILP

Row generation for the

TSP
TSP formulation has an exponential number of constraints.

The straight-forward application of branch-and-bound

method can process TSPs containing 40-60 cities.

Using branch-and-bound and problem-specific cut

generation, currently it can solve a TSP with 85,900 cities.

5.29
 Cutting Plane Method
An integer programming formulation

Suppose there are n cities. Let dij be the distance between

cities i and j, where dii = ∞. Define

1, if city j is reached from city i Introduction
xij =
0, otherwise. Method of Integer
Forms for IP

Then, the objective function of TSP can be written as Mixed cut for MILP

Row generation for the

n X
X n TSP
min z = dij xij
i=1 j=1

Constraints:
n
X
xij = 1,j = 1, 2, . . . , n
i=1
Xn
xij = 1,i = 1, 2, . . . , n
j=1

xij ∈ {0, 1},i, j = 1, 2, . . . , n

and
solution is a n-city tour. 5.30
 Cutting Plane Method
Subtour and tour

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

The left and center figures are subtours of the 6 cities, and are
not feasible solutions to the TSP.
The right figure is a tour of 6 cities, and is a feasible solution of
the TSP problem.

5.31
 Cutting Plane Method
Break the subtour

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

Let S = {1, 2, 3}. If they form a subtour, we should have

x12 + x21 + x13 + x31 + x23 + x32 = 3
If they don’t form a subtour,
x12 + x21 + x13 + x31 + x23 + x32 ≤ 3 − 1
⇒
X
xij ≤ |S| − 1 ∀S ⊂ {1, 2, . . . , n}
i6=j∈S 5.32
 Cutting Plane Method
Exercise

Write a subtour elimination constraint that can cuts off the

following solution.
Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.33
 Cutting Plane Method
Row generation for TSP

TSP formulation has an exponential number of constraints

(one for each proper subset of {1, 2, . . . , n})
Introduction
LP relaxation solution becomes unmanageable as n
Method of Integer
grows. Forms for IP

Mixed cut for MILP

Try relaxing (i.e., removing) problematic constraints Row generation for the
TSP
X
xij ≤ |S| − 1 ∀S ⊂ {1, 2, . . . , n}
i6=j∈S

⇒ TSP becomes an Assignment Problem and can be

solved very fast.
Algorithm:
1 The LP without subtour breaking constriants is solved.
2 If the LP relaxation solution is not a tour, a tour breaking
constraint is added, and the LP relaxation is solved again.
The branch-and-bound idea is used to break the sub-tours
one by one.
3 Keep going until all tour breaking constraints are satisfied by
the LP optimum.
5.34
 Cutting Plane Method
Example

Consider a 5-city TSP problem where the distance dij are given
by:
Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.35
 Cutting Plane Method
Example - subtour breaking

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP
The solution is {x12 = x21 = 1, x35 = x54 = x43 = 1} with
minimum cost z = 8.
Note that it is a solution with two subtours and hence is not a
feasible solution to the TSP.
In order to form a 5-city tour, we have to break either one of the
loops. Let’s break the loop 1 → 2 → 1.
⇒ either force x12 = 0 or x21 = 0
⇒ set d12 = ∞ or d21 = ∞

5.36
 Cutting Plane Method
Example - Enumeration tree

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.37
 Cutting Plane Method
Example - node 2

Set d12 = ∞. Solve the resulting assignment problem.

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

The solution is {x34 = x43 = 1, x15 = x52 = x21 = 1} with

minimum cost z = 9.
Contains subtours.
⇒ Next break the subtour 3 → 4 → 3.

5.38
 Cutting Plane Method
Example - Enumeration tree

Introduction

Method of Integer
Forms for IP

Mixed cut for MILP

Row generation for the

TSP

5.39
 Cutting Plane Method
Integer Programming Summary

Introduction
Dramatically improve the modeling capability. Method of Integer
Forms for IP

Economic indivisibility Mixed cut for MILP

Row generation for the

TSP
Logical constraint

Modeling non-linearities

Not as easy to model.

Not as easy to solve.

5.40
 Cutting Plane Method
IP Solution Techniques Summary

Introduction
Branch-and-Bound
Method of Integer
very general and adaptive Forms for IP

Mixed cut for MILP

used in practice (e.g. Excel solver) Row generation for the
TSP

Cutting planes (Valid inequalities)

clever way of improving bounding

used widely in practice

researchers continue to make improvements in this

approach

5.41