Lecture 9: Lagrangian dual problem

DSA3102 Essential Data Analytics Tools: Convex Optimisation

Lam Xin Yee

matlxy@[Link]
S17-08-11
In topic 5, we will study an equivalent formulation for solving constrained
optimization problem.
Lagrangian dual problems
Weak and strong duality theorems
Primal problem

We called the following NLP as a primal problem.

(P) min f (x)

s.t. gi (x) = 0, i = 1, 2, · · · , m
hj (x) ≤ 0, j = 1, 2, · · · , p
x ∈ X ⊆ Rn

Remark. The set constraint x ∈ X is to impose additional requirements

that we may wish to handle separately. Eg.
X = {x ∈ Rn | xi ≥ 0 ∀i = 1, 2, · · · , n}
X is a subset of Rn with integer components (for discrete problems
with integrality requirements).
X = Rn if there is no special requirement, and X will be omitted in
the formulation of the problem.
Lagrangian dual problem

(P) min f (x)

s.t. gi (x) = 0, i = 1, 2, · · · , m
hj (x) ≤ 0, j = 1, 2, · · · , p
x ∈ X ⊆ Rn

It might be difficult to solve (P) via KKT equations.

In this topic, we will study another NLP that is closely related to (P),
known as the Lagrangian dual problem (D), defined via the Lagrangian
function.

Solving (P) via (D) is sometimes easier than solving (P) directly.
Penalty for equality constraint
h i
min f (x) s.t. g1 (x) = 0 ≡ min max [f (x) + λ1 g1 (x)]
x∈X x∈X λ1 ∈R

Proposition 1
(
f (x), if g1 (x) = 0,
max [f (x) + λ1 g1 (x)] =
λ1 ∈R ∞, if g1 (x) 6= 0.
Penalty for equality constraint
h i
min f (x) s.t. g1 (x) = 0 ≡ min max [f (x) + λ1 g1 (x)]
x∈X x∈X λ1 ∈R

Proposition 1
(
f (x), if g1 (x) = 0,
max [f (x) + λ1 g1 (x)] =
λ1 ∈R ∞, if g1 (x) 6= 0.

Proof.
If x is feasible, then

g1 (x) = 0 =⇒ λ1 g1 (x) = 0 =⇒ f (x) + λ1 g1 (x) = f (x)

If x is infeasible, then
(
limλ→∞ λ1 g1 (x) = ∞, if g1 (x) > 0,
g1 (x) 6= 0 =⇒
limλ→−∞ λ1 g1 (x) = ∞, if g1 (x) < 0
=⇒ max [f (x) + λ1 g1 (x)] = ∞
λ1 ∈R
Penalty for inequality constraint
h i
min f (x) s.t. h1 (x) ≤ 0 ≡ min max [f (x) + µ1 h1 (x)]
x∈X x∈X µ1 ≥0

Proposition 2
(
f (x), if h1 (x) ≤ 0,
max [f (x) + µ1 h1 (x)] =
µ1 ≥0 ∞, if h1 (x) > 0.
Penalty for inequality constraint
h i
min f (x) s.t. h1 (x) ≤ 0 ≡ min max [f (x) + µ1 h1 (x)]
x∈X x∈X µ1 ≥0

Proposition 2
(
f (x), if h1 (x) ≤ 0,
max [f (x) + µ1 h1 (x)] =
µ1 ≥0 ∞, if h1 (x) > 0.

Proof.
If x is feasible, then
 
h1 (x) ≤ 0 =⇒ max [µ1 h1 (x)] = 0 =⇒ max f (x) + µ1 h1 (x) = f (x)
µ1 ≥0 µ1 ≥0

If x is infeasible, then
 
h1 (x) > 0 =⇒ lim µ1 h1 (x) = ∞ =⇒ max f (x) + µ1 h1 (x) = ∞
µ→∞ µ1 ≥0
General formulation

(P) min f (x)

s.t. gi (x) = 0, i = 1, 2, · · · , m
hj (x) ≤ 0, j = 1, 2, · · · , p
x ∈ X ⊆ Rn
(P) can be equivalently reformulated as
  
 m
X p
X 
min  max f (x) + λi gi (x) + µj hj (x) 
x∈X λ1 , ..., λm , µ1 ≥0, ..., µp ≥0  
i=1 j=1

or  
f (x) + λT g(x) + µT h(x)

min max ,
x∈X λ∈Rm , µ∈Rp+

where
       
g1 (x) h1 (x) λ1 µ1
g(x) =  ...  , h(x) =  ...  , λ =  ...  , µ =  ... 
       

gm (x) hp (x) λm µp
 " #
n o
T T
min max f (x) + λ g(x) + µ h(x)
x∈X p
λ∈Rm , µ∈R+

Let

L(x, λ, µ) = f (x) + λT g (x) + µT h(x),

and define the Lagrangian dual function:

θ(λ, µ) := infx∈X L(x, λ, µ) = inf{ f (x) + λT g(x) + µT h(x) : x ∈ X }

 " #
n o
T T
min max f (x) + λ g(x) + µ h(x)
x∈X p
λ∈Rm , µ∈R+

Let

L(x, λ, µ) = f (x) + λT g (x) + µT h(x),

and define the Lagrangian dual function:

θ(λ, µ) := infx∈X L(x, λ, µ) = inf{ f (x) + λT g(x) + µT h(x) : x ∈ X }

We arrived at the Lagrangian dual problem:

n o
T T
(D) max θ(λ, µ) or max inf f (x) + λ g(x) + µ h(x)
λ∈Rm , µ∈Rp+ λ∈Rm , µ∈Rp+ x∈X

λi , µj : Lagrangian dual variables or Lagrangian multipliers

(P) is the primal problem, and (D) is the dual problem
Outside syllabus: There are other duals (eg. Fenchel dual)
Example 9.1
Obtain the Lagrangian dual problem of the following nonlinear
programming problem (P).

min f (x) = x1 x2 x3
s.t. x1 − x2 + 5x3 = 6
x1 + 2x23 ≤ 4
(x1 − 3)3 − 4x22 + x3 ≤ 0
x12 + x22 − x3 ≤ 4
x ∈ R3 .
Example 9.1
Obtain the Lagrangian dual problem of the following nonlinear
programming problem (P).

min f (x) = x1 x2 x3
s.t. x1 − x2 + 5x3 = 6
x1 + 2x23 ≤ 4
(x1 − 3)3 − 4x22 + x3 ≤ 0
x12 + x22 − x3 ≤ 4
x ∈ R3 .

Solution. The Lagrangian dual problem (D) is:

max θ(λ, µ) = inf { x1 x2 x3 + λ(x1 − x2 + 5x3 − 6) + µ1 (x1 + 2x23 − 4)

+µ2 ((x1 − 3)3 − 4x22 + x3 ) + µ3 (x12 + x22 − x3 − 4) : x ∈ R3 }
s.t. λ ∈ R, µ1 , µ2 , µ3 ≥ 0.
 (P) min f (x)
s.t. g(x) = 0,
h(x) ≤ 0,
x ∈ X ⊆ Rn

n o
(D) max θ(λ, µ) := inf f (x) + λT g(x) + µT h(x)
λ∈Rm , µ∈Rp+ x∈X

In evaluating θ(λ, µ) for each λ, µ, the following problem (referred to

as the Lagrangian dual subproblem) is to be solved:

min f (x) + λT g(x) + µT h(x)

s.t. x ∈ X
Switching the order of minimization and maximization does not always
give the same result.
Example 9.2
h i
inf sup (e λ e x ) = inf (+∞) = +∞
x∈R λ∈R x∈R
h i
sup inf (e λ e x ) = sup (0) = 0
λ∈R x∈R λ∈R

In the next part, we will discuss a condition that makes this switching
equivalence, i.e. when solving the primal problem is the same as solving
the dual problem.
Geometrical interpretation of Lagrangian Dual Problem
Consider (P) with only one inequality constraint and no equality
constraint:
(P) min f (x) (P) min z2
s.t. h(x) ≤ 0 or s.t. z1 ≤ 0
x ∈ X. z = [z1 ; z2 ] ∈ G ,
where   
z1
G= : z1 = h(x), z2 = f (x), x ∈ X .
z2
This can be interpreted as minimizing the z2 -coordinate of points of G in
the left half plane.
 (P) min f (x) (P) min z2
s.t. h(x) ≤ 0 or s.t. z1 ≤ 0
x ∈ X. z = [z1 ; z2 ] ∈ G ,

The Lagrangian dual problem (D) of (P) is

(D) max θ(µ) = inf{f (x) + µh(x) : x ∈ X }

or
s.t. µ ≥ 0.
   
z1
(D) max θ(µ) = inf z2 + µz1 : ∈G
z2
s.t. µ ≥ 0.
 (P) min f (x) (P) min z2
s.t. h(x) ≤ 0 or s.t. z1 ≤ 0
x ∈ X. z = [z1 ; z2 ] ∈ G ,

The Lagrangian dual problem (D) of (P) is

(D) max θ(µ) = inf{f (x) + µh(x) : x ∈ X }

or
s.t. µ ≥ 0.
   
z1
(D) max θ(µ) = inf z2 + µz1 : ∈G
z2
s.t. µ ≥ 0.

For each fixed value c, each point (z1 , z2 ) on the line z2 + µz1 = c
has the same value z2 + µz1 , which is c. This value c is the
z2 -intercept of the line z2 + µz1 = c (this occurs when z1 = 0).
The value θ(µ) = inf{z2 + µz1 ; [z1 ; z2 ] ∈ G } is the intercept of the
support line to G (with slope −µ) on the z2 -axis.
To interpret the dual problem, consider the line z2 + µz1 with slope −µ
(µ ≥ 0) in the z1 -z2 plane. The dual problem is to find the slope of the
supporting line to G such that its intercept on the z2 -axis is the maximum.

From the diagram, the uppermost solid dot gives the solutions to both (P)
and (D).
Dual is concave
Proposition 3
Suppose the Lagrangian dual function

θ(λ, µ) = min{L(x, λ, µ) := f (x) + λT g(x) + µT h(x) | x ∈ X }

is finite for all (λ, µ) with µ ≥ 0. Then θ is a concave function.

Dual is concave
Proposition 3
Suppose the Lagrangian dual function

θ(λ, µ) = min{L(x, λ, µ) := f (x) + λT g(x) + µT h(x) | x ∈ X }

is finite for all (λ, µ) with µ ≥ 0. Then θ is a concave function.

Proof. Given (λ, b µ̄ ≥ 0. For α ∈ [0, 1], let

b (λ̄, µ̄) such that µ,
b µ),

(λ, µ) = α(λ, b + (1 − α)(λ̄, µ̄) = (αλ

b µ) b + (1 − α)λ̄, αµ
b + (1 − α)µ̄).
Then
L(x, λ, µ) = f (x) + λT g(x) + µT h(x)
b T g(x) + µ
= α[f (x) + λ b T h(x)] + (1 − α)[f (x) + λ̄T g(x) + µ̄T h(x)]

= αL(x, λ, b + (1 − α)L(x, λ̄, µ̄)

b µ)

≥ αθ(λ, b + (1 − α)θ(λ̄, µ̄) ∀ x ∈ X .

b µ)

=⇒ θ(λ, µ) = min{L(x, λ, µ) : x ∈ X } ≥ αθ(λ, b + (1 − α)θ(λ̄, µ̄).

b µ)
Example 9.3
Solve the following convex program and its Lagrangian dual problem.

min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | x1 , x2 ≥ 0
x2
 
2
Solution. Note that the optimal solution occurs at x∗ = , and
2
f (x∗ ) = 8 which can be obtained graphically.
Example 9.3
Solve the following convex program and its Lagrangian dual problem.

min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | x1 , x2 ≥ 0
x2
 
2
Solution. Note that the optimal solution occurs at x∗ = , and
2
f (x∗ ) = 8 which can be obtained graphically.

For each µ ≥ 0, the Lagrangian dual function is

θ(µ) = inf {x12 + x22 + µ(−x1 − x2 + 4)|x1 , x2 ≥ 0}

= inf{x12 − µx1 | x1 ≥ 0} + inf{x22 − µx2 |x2 ≥ 0} + 4µ .
2
= − µ2 + 4µ, attained when x1 = µ2 , x2 = µ2 .
Example 9.3 (cont’d)

Thus, the dual problem (D) is

2
max θ(µ) = − µ2 + 4µ
s.t. µ ≥ 0
Note that the objective function θ(µ) is a concave quadratic function. The
optimal solution occurs at µ∗ = 4, and θ(µ∗ ) = 8.
µ∗ µ∗
Notice that the value θ(µ∗ ) is attained when x1∗ = 2 = 2, x2∗ = 2 = 2.

Observation. Optimal value for (P) = optimal value for (D).

Example 9.4
Solve the following convex program and its Lagrangian dual problem.

(P) min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | 0 ≤ x1 ≤ 1, 1 ≤ x2 ≤ 5 .
x2

Solution.
  Graphically, the optimal solution of (P) occurs at
1
x∗ = and f (x∗ ) = 10.
3
Example 9.4
Solve the following convex program and its Lagrangian dual problem.

(P) min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | 0 ≤ x1 ≤ 1, 1 ≤ x2 ≤ 5 .
x2

Solution.
  Graphically, the optimal solution of (P) occurs at
1
x∗ = and f (x∗ ) = 10.
3

For each µ ≥ 0, the Lagrangian dual function θ(µ) is

θ(µ) = inf{x12 + x22 + µ(−x1 − x2 + 4) | [x1 ; x2 ] ∈ X }

= inf{x12 − µx1 | 0 ≤ x1 ≤ 1} + inf{x22 − µx2 | 1 ≤ x2 ≤ 5} + 4µ.
Example 9.4 (cont’d)
Next we find the minimum values for x12 − µx1 and x22 − µx2 , they occur at

1 if 0 ≤ µ ≤ 2
 
µ
 2 if 0 ≤ µ ≤ 2 

µ
x1 =
 1 if µ > 2
, x2 =
 2 if 2 ≤ µ ≤ 10

5 if µ > 10

Therefore,
 2
 −µ if 0 ≤ µ ≤ 2
4
inf{x12 − µx1 |0 ≤ x1 ≤ 1} =
 1 − µ if µ > 2

and


 1−µ if 0 ≤ µ ≤ 2

2
inf{x22 − µx2 |1 ≤ x2 ≤ 5} = − µ4 if 2 ≤ µ ≤ 10


25 − 5µ if µ > 10

Example 9.4 (cont’d)
Thus, the Lagrangian dual function is
 2

 (− µ4 ) + (1 − µ) + 4µ if 0≤µ≤2

2
θ(µ) =
 (1 − µ) + (− µ4 ) + 4µ if 2 ≤ µ ≤ 10

(1 − µ) + (25 − 5µ) + 4µ

if µ > 10
( 2
− µ4 + 3µ + 1 if 0 ≤ µ ≤ 10
=
−2µ + 26 if µ > 10
Notice that θ(µ) is a concave function. Example 6.3: θ(μ)
The Lagrangian dual problem (D) is 15

10

5
max{θ(µ) : µ ≥ 0}.
0

Graphically, the optimal solution of D oc- −5

curs at µ∗ = 6 and θ(µ∗ ) = 10. −10

−15
0 5 10 15 20
Example 9.5 (Dual problem includes primal variables)
Find the dual problem implicitly of the following problem

min f (x) := exp(x)

s.t. x 2 − 1 ≤ 0,

Solution. For µ ≥ 0, the Lagrangian dual function

θ(µ) = inf exp(x) + µ(x 2 − 1) | x ∈ R .


Example 9.5 (Dual problem includes primal variables)
Find the dual problem implicitly of the following problem

min f (x) := exp(x)

s.t. x 2 − 1 ≤ 0,

Solution. For µ ≥ 0, the Lagrangian dual function

θ(µ) = inf exp(x) + µ(x 2 − 1) | x ∈ R .



If µ = 0, then θ(µ) = 0. The function exp(x) + µ(x 2 − 1) is convex on R,

hence any stationary point will be a global minimizer. This yields a
stationary point x ∗ (dependent on µ) which satisfies exp(x ∗ ) + 2µx ∗ = 0 if
µ > 0.

For each µ > 0, there is no analytical solution for x ∗ (in fact x ∗ cannot be
expressed explicitly in terms of µ).
Example 9.5 (cont’d)

(
exp(x) + µ(x 2 − 1) with exp(x) + 2µx = 0 if µ > 0
Thus, θ(µ) =
0 if µ = 0
Therefore, the dual problem (D) is

max θ(µ) = exp(x) + µ(x 2 − 1)

s.t. exp(x) + 2µx = 0, µ > 0, x ∈ R.
Example 9.6 (Linear programs)
Consider the standard form primal linear programming problem.

(LP) min f (x) := cT x

subject to b − Ax = 0,
x ≥ 0, x ∈ Rn .

where A is an m × n matrix and b ∈ Rm .

(a) Determine the Lagrangian dual function.
(b) Determine the Lagrangian dual problem.

Solution. Note: (LP) is a convex program. Let X = {x ∈ Rn : x ≥ 0} and

λ ∈ Rm .
(a) The Lagrangian dual function is

θ(λ) = inf{cT x + λT (b − Ax) | x ∈ X }

= λT b + inf{xT (c − AT λ) | x ∈ X }
(
λT b if c − AT λ ≥ 0
=
−∞ otherwise
Example 9.6 (cont’d)

(b) Hence the Lagrangian dual problem can be stated as follows

max θ(λ) = λT b
s.t. c − AT λ ≥ 0,
λ ∈ Rm

which is the dual LP of the given standard form LP.

For a linear programming problem (P), the above Lagrangian dual

problem (D) is the same dual LP which we have discussed in MA3252
(Linear and networking optimization).
Example 9.7 (Quadratic program)
Find the dual problem of the quadratic program

min f (x) := 12 xT Qx + cT x
s.t. Ax ≤ b,
x ∈ Rn

where Q is a symmetric positive semidefinite matrix (hence may not have Q −1 )

and b ∈ Rm .
Example 9.7 (Quadratic program)
Find the dual problem of the quadratic program

min f (x) := 12 xT Qx + cT x
s.t. Ax ≤ b,
x ∈ Rn

where Q is a symmetric positive semidefinite matrix (hence may not have Q −1 )

and b ∈ Rm .
The Lagrangian dual function is
θ(µ) = inf{L(x, µ) := 12 xT Qx + cT x + µT (Ax − b) | x ∈ Rn }
For a given µ ≥ 0, the function L(x, µ) is a convex function of x, and achieves its
minimum at a stationary point x of L(x, µ), i.e. ∇x L(x, µ) = 0.
Qx + AT µ + c = 0.
Thus the dual problem can be stated as follows
1 T
max 2 x Qx + cT x + µT (Ax − b)
s.t. Qx + AT µ + c = 0
In topic 5, we will study an equivalent formulation for solving constrained
optimization problem.
Lagrangian dual problems
Weak and strong duality theorems
Weak duality theorem

How is the dual connected with the primal?

Theorem 9.8 (Weak Duality Theorem)

Consider the primal and dual pair (P) and (D). Let x be a feasible solution
to (P) and (λ, µ) be a feasible solution to (D). Then

f (x) ≥ θ(λ, µ).

Weak duality theorem

How is the dual connected with the primal?

Theorem 9.8 (Weak Duality Theorem)

Consider the primal and dual pair (P) and (D). Let x be a feasible solution
to (P) and (λ, µ) be a feasible solution to (D). Then

f (x) ≥ θ(λ, µ).

Proof. By the definition of θ(λ, µ), we have

θ(λ, µ) = inf{ f (y) + λT g(y) + µT h(y) | y ∈ X }

≤ f (x) + λT g(x) + µT h(x)
≤ f (x), since g(x) = 0, µ ≥ 0, h(x) ≤ 0.
Example 9.9 (Revisit example 9.4)
Verify the weak duality for the following primal NLP

(P) min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | 0 ≤ x1 ≤ 1, 1 ≤ x2 ≤ 5
x2

and its Lagrangian dual problem

(D) max{θ(µ) : µ ≥ 0}
( 2
− µ4 + 3µ + 1 if 0 ≤ µ ≤ 10
where θ(µ) =
−2µ + 26 if µ > 10
 
1
At a primal feasible point b
x= , f (b
x) = 26.
5
At a dual feasible point µ̂ = 8, θ(µ̂) = 9.
 (P) min f (x)
s.t. g(x) = 0, (D) max p
θ(λ, µ)
λ∈Rm , µ∈R+
h(x) ≤ 0, := inf x∈X f (x) + λT g(x) + µT h(x)


x ∈ X ⊆ Rn

Corollary 9.10
(a) Optimal primal (minimization) objective value ≥ Optimal dual
(maximization) objective value. I.e.,

min{f (x) : x ∈ S} ≥ max{θ(λ, µ) : λ ∈ Rm , µ ≥ 0}.

 (P) min f (x)
s.t. g(x) = 0, (D) max p
θ(λ, µ)
λ∈Rm , µ∈R+
h(x) ≤ 0, := inf x∈X f (x) + λT g(x) + µT h(x)


x ∈ X ⊆ Rn

Corollary 9.10
(a) Optimal primal (minimization) objective value ≥ Optimal dual
(maximization) objective value. I.e.,

min{f (x) : x ∈ S} ≥ max{θ(λ, µ) : λ ∈ Rm , µ ≥ 0}.

(b) If x∗ is a feasible solution to (P) and (λ∗ , µ∗ ) is a feasible solution to (D)

such that
f (x∗ ) = θ(λ∗ , µ∗ )
then x∗ is an optimal solution to (P) and (λ∗ , µ∗ ) is an optimal solution to
(D).
Duality gap

(P) min f (x)

s.t. g(x) = 0, (D) max p
θ(λ, µ)
λ∈Rm , µ∈R+
h(x) ≤ 0, := inf x∈X f (x) + λT g(x) + µT h(x)


x ∈ X ⊂ Rn

Weak Duality Theorem: f (x) ≥ θ(λ, µ).

Suppose x∗ is optimal for (P) and (λ∗ , µ∗ ) is optimal for (D).

For any primal feasible x and dual feasible (λ, µ),
f (x) ≥ f (x∗ ) ≥ θ(λ∗ , µ∗ ) ≥ θ(λ, µ)
The Lagrangian dual problem (D) seeks to search for the greatest
lower bound for f (x∗ ).
The difference
f (x∗ ) − θ(λ∗ , µ∗ ) := min{f (x) : x ∈ S} − max{θ(λ, µ) : µ ≥ 0}.
is called the duality gap.
Example 9.11 (Revisit example 9.4)
The duality gap of the following NLP

(P) min f (x) := x12 + x22

s.t. h(x) := −x1 − x2 + 4 ≤ 0,
  
x1
x∈X = | 0 ≤ x1 ≤ 1, 1 ≤ x2 ≤ 5
x2

and its dual is zero, i.e. there is no duality gap.

In example 9.4, the primal and dual objective values are

f (x∗ ) = 10, θ(µ∗ ) = 10.

Example 9.12
Consider the following NLP:

min f (x) = −2x1 + x2

+ x2 −
s.t. g (x) = x1  3= 0        
0 0 4 4 1 2
x∈X = , , , , ,
0 4 4 0 2 1

Find the dual problem and the duality gap.

Note: this problem is not convex.

There are only 6 points in X , thus it can be checked

  that the optimal
2
primal objective value is f (x∗ ) = −3 and x∗ = .
1
Example 9.12 (cont’d)

The Lagrangian dual function is

θ(λ) = inf{ f (x) + λT g (x) : x ∈ X }

= inf{−3λ + (λ − 2)x1 + (λ + 1)x2 : x ∈ X }
= inf{−3λ, λ + 4, 5λ − 4, λ − 8, 0, −3}
5λ − 4 if λ ≤ −1



= λ − 8 if − 1 ≤ λ ≤ 2


−3λ if λ ≥ 2

The maximum value of θ(λ) occurs at λ∗ = 2 with θ(λ∗ ) = −6 (by

sketching the graph).

The duality gap is f (x∗ ) − θ(λ∗ ) = −3 − (−6) = 3.

Strong Duality Theorem
Under certain convexity assumptions and suitable constraint qualifications,
the primal and dual problems have equal optimal objective value and
hence it is possible to solve the primal problem indirectly by solving the
dual problem (which is sometimes easier).
Theorem 9.13 (Strong Duality Theorem)
Consider the primal NLP (P), where
X is a convex set, f , hj , j = 1, . . . , p are convex functions, and gi ,
i = 1, . . . , m are affine functions.
Slater’s condition holds, i.e. there exists b x ∈ X such that h(bx) < 0
and g (bx) = 0
Then the duality gap is zero, i.e.

inf{f (x) : g(x) = 0, h(x) ≤ 0, x ∈ X } = sup{θ(λ, µ) : µ ≥ 0, λ ∈ Rm }.

Furthermore, if inf in (P) is finite, then sup is attained at some (λ∗ , µ∗ ).

If inf is attained at x∗ , then µT ∗
∗ h(x ) = 0.
Example 9.14 (Strong duality might fail without Slater’s condition)
Consider
min exp(−x1 )
x12
s.t. x2 +1≤ 0,
x ∈ X := {x ∈ R2 : x2 ≥ 0}.
Show that the strong duality fails.

Solution. Note that this is a convex problem, but x1 is always 0 in the

feasible region. Hence, Slater’s condition does not hold.

The minimum of the primal problem is 1 when x1∗ = 0. The Lagrangian

dual function is
(
n x12 o 0, when µ ≥ 0,
θ(µ) = inf exp(−x1 ) + µ : x ∈X =
x2 + 1 −∞, otherwise

Hence the maximum value of θ(µ) for µ ≥ 0 is 0. The duality gap is 1.

Flow chart provided by Tong Xin
Example 9.15
Consider the (1-variable problem):

(P) min f (x) := −x 2

s.t. −x ≤ 0,
x ∈ X := {x ∈ R : − 12 ≤ x ≤ 1}

(a) Find an optimal solution and optimal objective value of the primal
problem (P).
(b) Formulate and solve the dual of (P).

Solution. x ∗ = 1 and f (x ∗ ) = −1. θ(µ) = −1 − µ.