END508E Mathematical

Optimization

Class 1: Introduction to Mathematical Optimization

Source (Bertsimas and Tsitsiklis)
History of Optimization
• Fermat, Newton (1600s), Euler, Lagrange and more
minimize f(𝑥)
𝑥 ∈ ℛ𝑛

minimize f(𝑥1 , 𝑥2 , … , 𝑥𝑛 )
𝑔𝑘 𝑥1 , 𝑥2 , … , 𝑥𝑛 = 0
𝑘 = 1, … , 𝑚

END508E Mathematical Optimization 2

General Optimization Model
m𝑖𝑛 𝑓 𝑥1 , 𝑥2 , 𝑥3 , … , 𝑥𝑛
s.t.
𝑔1 𝑥1 , 𝑥2 , 𝑥3 , … , 𝑥𝑛 ≤ 0
𝑔2 𝑥1 , 𝑥2 , 𝑥3 , … , 𝑥𝑛 ≤ 0
....
𝑔𝑚 𝑥1 , 𝑥2 , 𝑥3 , … , 𝑥𝑛 ≤ 0

• Too general to solve, must specify properties of X, f and g more

precisely.

END508E Mathematical Optimization 3

General Optimization Model
• An optimization problems concerns:
• A set of decisions to make
• A single quantitative criterion to compare decisions (in cases of single objective)
• Rules governing interactions of decisions
Example: Diet problem
• Decisions: What typical food shall we eat? How much shall we eat?
• Criterion: We must earn enough energy, protein, fat, glucose, vitamins. OR a
minimum cost diet must be reached.
• Rules: Total energy should not exceed our requirement. At least some protein,
fat, vitamins should be collected.
OR the food cost should not exceed our budget.

END508E Mathematical Optimization 4

General Optimization Model
• A Mathematical Program (or Mathematical Optimization Problem in
modern terms) is a mathematical representation of an optimization
problem in which some decisions must be made in order to reach the
best value of a criterion (obj. Function), by satisfying the predefined
rules.
• Decisions  Decision variables
• Comparison criterion  Objective function
• Rules governing the interactions  Constraints

END508E Mathematical Optimization 5

Linear Optimization

END508E Mathematical Optimization 6

Standard form

• Elimination of free variables:

• Elimination of inequality constraints:

END508E Mathematical Optimization 7

History of Linear Optimization
• Pre-algorithmic period:
• Fourier, 1826: Methods for solving system of linear inequalities
• de la Vallee Poussin: simplex like method for objective function with absolute values
• Kantorovich and Koopmans, 1930: resource allocation model and solution method
• von Neumann, 1928: game theory and duality
• Farkas and Minkowski: foundations
• Modern period:
• George Danztig, 1947: Simplex algorithm
• 50s: applications
• 60s: large scale optimization
• 70s: complexity theory and ellipsoid algorithm
• 80s: interior point algorithm
• 90s: semidefinite and conic optimization
• 2000s: robust optimization
END508E Mathematical Optimization 8
Examples
• Knapsack problem: There is a knapsack (bag). There are a number of
items, each with a size and a value. The objective is to maximize the
total value of the items in the bag.

END508E Mathematical Optimization 9

Production Problem
A firm produces 𝑛 different goods using 𝑚 different raw materials. Let
𝑏𝑖 , 𝑖 = 1, . . . , 𝑚, be the available amount of the ith raw material. The
jth good, 𝑗 = 1, … , 𝑛, requires 𝑎𝑖𝑗 units of the ith material and results
in a revenue of 𝐶𝑗 per unit produced. The firm faces the problem of
deciding how much of each good to produce in order to maximize its
total revenue.

END508E Mathematical Optimization 10

Scheduling problem
A hospital wants to make a weekly night shift ( 12pm-8am) schedule for
its nurses . The demand for nurses for the night shift on day 𝑗 is an
integer 𝑑𝑗 , 𝑗 = 1 , . . . , 7. Every nurse works 5 days in a row on the
night shift . The problem is to find the minimal number of nurses the
hospital needs to hire.

END508E Mathematical Optimization 11

Problems with absolute values

END508E Mathematical Optimization 12

 Graphical representation and solution
• Consider the problem:

• For any given scalar z, we assume Feasible

• cTx=z. -x1-x2=z. region
• This line is perpendicular to the vector c=(-1, -1).
• Different z values lead to different lines, all parallel to each
other.
• As we min the obj, we move the line -x1-x2=z along the
direction of vector c.
• Z=-2 : Best obj value. X=(1, 1) optimal solution.
• This is the corner of the feasible set!!! (Attention!!)

Assoc. Prof. Sule I. Satoglu 13

Graphical representation and solution
• For a problem in three dimensions, the same approach can be used except
that the set of points with the same value of cTx is a plane, instead of a line.
• This plane is again perpendicular to the vector c, and the objective is to
slide this plane as much as possible in the direction of -c, as long as we’re
in the feasible set.

END508E Mathematical Optimization 14

 Graphical representation and solution
• Example: Suppose that the feasible set is
the unit cube, described by the constraints
• 0<=xi<=1; i=1, 2, 3. c=(-1, -1, -1)
• X=(1,1,1) is the opt solution.
• Once again, the opt solution is in the
corner of the feasible solution set.

END508E Mathematical Optimization 15

Graphical representation and solution
• The feasible set is bounded (does not extend to infinity), and the
problem has a unique optimal solution.
• This is not always the case.
• Some times the problem is unbounded, or infeasible or has multiple
solutions.

END508E Mathematical Optimization 16

 Graphical representation and solution
• Consider the feasible set in R2.

1. For the cost vector c=(1,1), x=(0,0) is the unique opt

solution.
2. For the cost vector c = (1,0), there are multiple optimal
solutions, namely, every vector x of the form x = (0, X2),
with 0 <=X2<=1, is optimal. Note that the set of optimal
solutions is bounded.
3. For the cost vector c = (0, 1), there are multiple optimal
solutions. X=(x1, 0) is the opt solution for each x1>=0. In
this case, the set of optimal solutions is unbounded.
4. For c = (-1, -1). For any (X1, X2), we can always produce
another feasible solution with less cost, by increasing the
value of X1. Therefore, no feasible solution is optimal. Opt
solution is -.
END508E Mathematical Optimization 17
Graphical representation and solution
• So, we have the following possibilities:
• (a) There exists a unique optimal solution.
• (b) There exist multiple optimal solutions; in this case, the set of
optimal solutions can be either bounded or unbounded.
• (c) The optimal cost is -, and no feasible solution is optimal.
• (d) The feasible set is empty.
• If the LP problem has at least one optimal solution, then an optimal
solution can be found among the corners of the feasible set.
• This is a general feature of linear programming problems, as long as
the feasible set has at least one corner.
END508E Mathematical Optimization 18