Scribd
Upload
37 views12 pages

kkt_condition_NLP

Uploaded by

Giridhar meda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
37 views12 pages

kkt_condition_NLP

Uploaded by

Giridhar meda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

KKT Condition for NLP with

inequality constraints
KKT Condition (Karush-Kuhn-Tucker condition)

NLP with inequality constraints


𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑓(𝑋)
Non-linear and
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜:
continuous
𝑔𝑗 𝑋 ≤ 0, ∀ 𝑗 = 1, … . . 𝑚
constraints

n dimensional 𝑋 = 𝑥1 𝑥2 … … 𝑥𝑛 𝑇 ∈ 𝑅𝑛
vector of
decision
variables
Step 1: Transform the given problem into Lagrangian function

• Introduce 𝑚 slack variables 𝑆𝑗

• The inequality constraints are transformed as

𝑔𝑗 𝑋 + 𝑆𝑗2 = 0 , ∀ 𝑗 = 1, … . . , 𝑚

• Lagrangian function is

𝐿 𝑥1 , 𝑥2 , … 𝑥𝑛 , 𝑆1 , 𝑆2 , … , 𝑆𝑚 , 𝜆1 , 𝜆2 , … , 𝜆𝑚 =

𝑓 𝑋 + σ𝑚 2
𝑗=1 𝜆𝑗 (𝑔𝑗 𝑋 + 𝑆𝑗 ) -----------------------------------(1)
Step 2: Develop set of necessary conditions ( gives local / relative minima)

• Partial derivative with respect to decision variables is equal to zero


𝜕𝐿ൗ
𝜕𝑥𝑘 = 0, ∀ 𝑘 = 1,2, … . , 𝑛

𝜕𝑓 𝑚 𝜕𝑔𝑗 (𝑋)
ൗ𝜕𝑥𝑘 + σ𝑗=1 𝜆𝑗 = 0 , ∀ 𝑘 = 1, … . . , 𝑛 --------(2)
𝜕𝑥𝑘

• Partial derivative with respect to lagrange multiplier is equal to zero

𝜕𝐿ൗ
𝜕𝜆𝑗 = 0, ∀𝑗 = 1, … , 𝑚
𝑔𝑗 𝑋 + 𝑆𝑗2 = 0, ∀ 𝑗 = 1, … . , 𝑚 ---------------(3)
Step 2: Develop set of necessary conditions ( given local / relative minima)

• Partial derivative with respect to slack variables is equal to zero

𝜕𝐿ൗ
𝜕𝑆𝑗 = 0, ∀ 𝑗 = 1,2, … . , 𝑚
𝜆𝑗 × 𝑆𝑗 = 0, ∀ 𝑗 = 1, … , 𝑚-------------------(4)
• Equation 2, 3, and 4 together are known as KKT Conditions
• Now coming to equation 4, there are two possibilities of 𝜆𝑗 and 𝑆𝑗
values
• 𝜆𝑗 = 0, 𝑆𝑗 ≠ 0 , set of inactive constraints
• 𝜆𝑗 ≠ 0, 𝑆𝑗 = 0, set of active constraints
Step 2: Develop set of necessary conditions ( given local / relative minima)C

• Thus in general, KKT Condition includes

𝜕𝑓 𝑚 𝜕𝑔𝑗 (𝑋)
1. ൗ𝜕𝑥𝑘 + σ𝑗=1 𝜆𝑗
𝜕𝑥𝑘
= 0 , ∀ 𝑘 = 1, … . . , 𝑛 OPTIMALITY CONDITION

2. 𝑔𝑗 𝑋 ≤ 0, ∀ 𝑗 = 1, … 𝑚 FEASIBILITY CONDITION

3. 𝜆𝑗 × 𝑔𝑗 𝑋 = 0, ∀𝑗 COMPLEMENTARY SLACKNESS PROPERTY

4. 𝜆𝑗 ≥ 0, ∀𝑗 NON-NEGATIVITY CONSTRAINTS

If 𝑋 ∗ satisfies all of these condition then it is a local or relative minima


Step 3: Solve considering different possibility of active and in-active constraints

• For a problem with two inequality constraints, there will be four possibilities
• 𝜆1 = 0, 𝜆2 = 0, inactive constraints
• 𝜆1 ≠ 0, 𝜆2 = 0, or 𝜆1 = 0, 𝜆2 ≠ 0, active and inactive constraints
• 𝜆1 ≠ 0, 𝜆2 ≠ 0, active constraints
Step 4: Check for sufficient condition for optimality
KKT Conditions are sufficient and will give GLOBAL OPTIMA if the objective
function is convex (minimization) or concave (maximization) and the feasible
region is also convex i.e. For a Convex programming problem
Problem on KKT Condition
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑓 𝑋 = 𝑥12 − 4𝑥1 + 𝑥22 − 6𝑥2
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜:
𝑥1 + 𝑥2 ≤ 3
−2𝑥1 + 𝑥2 ≤ 2
𝑥1 , 𝑥2 ≥ 0
Step 1: Transform into Lagrangian function
𝐿 𝑋, 𝜆 = 𝑥12 − 4𝑥1 + 𝑥22 − 6𝑥2 + 𝜆1 𝑥1 + 𝑥2 − 3 + 𝜆2 (−2𝑥1 + 𝑥2 − 2)

Step 2: From Optimality condition


2𝑥1 − 4 + 𝜆1 − 2𝜆2 = 0 ---------(1) 2𝑥2 − 6 + 𝜆1 + 𝜆2 = 0 ---------(2)
From feasibility condition
𝑥1 + 𝑥2 ≤ 3--------------(3) −2𝑥1 + 𝑥2 ≤ 2 ----------------(4)
From Complementary slackness
𝜆1 × 𝑥1 + 𝑥2 − 3 = 0 --------(5) 𝜆2 −2𝑥1 + 𝑥2 − 2 = 0 ---------(6)
Problem on KKT Condition

Step 2: From Optimality condition


2𝑥1 − 4 + 𝜆1 − 2𝜆2 = 0 ---------(1) 2𝑥2 − 6 + 𝜆1 + 𝜆2 = 0 ---------(2)
From feasibility condition
𝑥1 + 𝑥2 ≤ 3--------------(3) −2𝑥1 + 𝑥2 ≤ 2 ----------------(4)
From Complementary slackness
𝜆1 × 𝑥1 + 𝑥2 − 3 = 0 --------(5) 𝜆2 −2𝑥1 + 𝑥2 − 2 = 0 ---------(6)
From Non-negativity constraints
𝜆1 , 𝜆2 ≥ 0
Problem on KKT Condition

• For a problem with two inequality constraints, there will be four possibilities

• Case 1: 𝜆1 = 0, 𝜆2 = 0, inactive constraints

• Case 2: 𝜆1 ≠ 0, 𝜆2 = 0,
one active and one
inactive constraints
• Case 3: 𝜆1 = 0, 𝜆2 ≠ 0,

• Case 4: 𝜆1 ≠ 0, 𝜆2 ≠ 0, both active constraints


Problem on KKT Condition
• Case 1: 𝜆1 = 0, 𝜆2 = 0, gives solution for 𝑥1 , 𝑥2 using Equation 1 and 2
2𝑥1 − 4 = 0 and 2𝑥2 − 6 = 0 as 𝑥1 = 2, 𝑥2 = 3
These does not satisfy the feasibility condition in Equation 3 and are not
optimal

• Case 2: 𝜆1 ≠ 0, 𝜆2 = 0, gives solution for 𝑥1 , 𝑥2 using Equation 1,2, and 5


as 𝑥1 = 1, 𝑥2 = 2, 𝜆1 = 2
These satisfy all the conditions and could be a KKT Point or optimal solution
Problem on KKT Condition
• Case 3: 𝜆1 = 0, 𝜆2 ≠ 0, gives solution for 𝑥1 , 𝑥2 using Equation 1, 2, and 6
4 18
as 𝑥1 = , 𝑥2 = , 𝜆2 = −6/5
5 5

These does not satisfy the non-negativity condition are not optimal

• Case 4: 𝜆1 ≠ 0, 𝜆2 ≠ 0, gives solution for 𝑥1 , 𝑥2 using Equation 1,2, 5 and 6


1 8 14 8
as 𝑥1 = , 𝑥2 = ,𝜆 = , 𝜆2 = −
3 3 1 9 9

These does not satisfy the non-negativity condition are not optimal

You might also like