Overview of Optimization

Topics

• Optimization
• Objective function
• Decision Variables
• Solutions for optimization problem
• Decision space
• Constraints or restrictions
• State variables
• Local and Global optima
• Near Optimal solutions
• Simulations
Optimization
• Engineers are commonly confronted with the tasks of designing and
operating systems to meet or surpass specified goals while meeting
numerous constraints imposed on the design and operation.
• Optimization is the organized search for such designs and operating
modes.
• It determines the set of actions or elements that must be
implemented to achieve optimized systems.
• In the simplest case, optimization seeks the maximum or minimum
value of an objective function corresponding to variables defined in a
feasible range or space.
Optimization (Contd…)
More generally, optimization is the search of the set of variables that produces the
best values of one or more objective functions while complying with multiple
constraints.
A single‐objective optimization model embodies several mathematical expressions
including an objective function and constraints as follows:
 Objective Function
 The objective function constitutes the goal of an optimization

problem.

 That goal could be maximized or minimized by choosing variables,

or decision variables, that satisfy all problem constraints.

 The desirability of a set of variables as a possible solution to an

optimization problem is measured by the value of objective function

corresponding to a set of variables.

 Objective Function (CONTD…)
 Some of the algorithms with optimization problems that involve

maximizing the objective function.

 Others do so with optimization problems that minimize the objective

function.

 A maximization (or minimization) problem can be readily converted,

if desired, to a minimization (or maximization) problem by multiplying

its objective function by −1.

Decision Variables
• The decision variables determine the value of the objective function.

• In each optimization problem we search for the decision variables that

yield the best value of the objective function or optimum.

• In some optimization problems the decision variables range between an

upper bound and a lower bound. This type of decision variables forms a

continuous decision space.

• On the other hand, there are optimization problems in which the decision

variables are discrete. Discrete decision variables refer to variables

that take specific values between an upper bound and a lower bound.
Decision Variables (CONTD…)
• Optimization problems involving continuous decision variables are

called continuous problems, and those problems defined in terms of

discrete decision variables are known as discrete problems.

• There are, furthermore, optimization problems that may involve discrete

and continuous variables. One such example would be an optimization

involving the decision of whether or not to build a facility at a certain

location and, if so, what its capacity ought to be. The siting variable is of

the binary type (0 or 1), whereas its capacity is a real, continuous

variable. This type of optimization problem is said to be of mixed type.

Solutions of an Optimization Problem

• Each objective function is expressed in terms of decision

variables.

• When there is only one decision variable, the optimization

problem is said to be one‐dimensional, while optimization
problems with two or more decision variables are called N‐
dimensional.

• An N‐dimensional optimization problem has solutions that

are expressed in terms of one or more sets of solutions in
which each solution has N decision variables.
Decision Space

• The set of decision variables that satisfy the constraints of an

optimization problem is called the feasible decision space.

• In an N‐dimensional problem, each possible solution is an N‐vector

variable with N elements.

• Each element of this vector is a decision variable.

• Optimization algorithms search for a point (i.e., a vector of

decision variables) or points (i.e., more than one vector of decision
variables) in the decision space that optimizes the objective
function.
Constraints or Restrictions
• Each optimization problem may have two types of constraints.
• Some constraints directly restrict the possible value of the
decision variables, such as a decision variable x being a positive real
number, x > 0.
• Another form of constraint is written in terms of formulas, such as
when two decision variables x1 and x2 are restricted to the space x1
+ x2 ≤ b.
• The goal of an optimization problem is to find an optimal feasible
solution that satisfies all the constraints and yields the best
value of the objective function among all feasible solutions.
Figure 1 depicts a constrained two‐dimensional decision space with
infeasible and feasible spaces.

Figure. 1 : Decision space of a constrained two ‐dimensional optimization problem.

The set of all feasible solutions constitute the feasible decision space,
and the infeasible decision space is made up of all the infeasible
decision variables. Evidently, the optimal solution must be in the
State variables
• State variables are dependent variables whose values
change as the decision variables change their values.

• State variables are important in engineering problems

because they describe the system being modeled and the
objective function and constraints are evaluated
employing their values.

• As an example, consider an optimization problem whose

objective is to maximize hydropower generation by
operating a reservoir.
State variables
• The decision variable is the amount of daily water release
passing through turbines.

• The state variable is the amount of water stored in the

reservoir, which is affected by the water released through
turbines according to an equation of water balance that
also involves water inflow to the reservoir, evaporation from
the reservoir, water diversions or imports to the reservoir,
water released from the reservoir bypassing turbines, and
other water fluxes that change the amount of reservoir
Local and Global Optima
 A local optimum refers to a solution that has the best objective function in its

neighborhood

 In a one‐dimensional optimization problem, a feasible decision variable X* is a

local optimum of a maximization problem if the following condition holds:

 In a minimization problem the local‐optimum condition becomes

 where = a local optimum and ɛ = limited length in the neighborhood about the
local optimum. A local optimum is limited to a neighborhood of the decision space,
and it might not be the best solution over the entire decision space.
 Local and Global Optima
 A global optimum is the best solution in the decision space.

 Some optimization problems may have more than one—in fact,

an infinite number of global optima. These situations arise

commonly in linear programming problems.

 In this case, all the global optima produce the same value of

the objective function. There are not decision variables that

produce a better objective function value than the global
optimum.
Local and Global Optima
 A one‐dimensional optimization problem with decision variable X and

objective function f(X) the value X* is a global optimum of a

maximization problem if for any decision variable X the following is
true:

 In a minimization problem we would have

 Local and Global Optima
 Figure 2 illustrates global and local optima for a one‐dimensional maximization

problem.

L1, L2, and L3 in Figure 2 are local optima, and G denotes the global optimum with the largest value of the
objective function.
The decision space may be single modal or multimodal.

In a single‐modal surface, there is only one extreme point, while there are
several extremes on the surface of a multimodal problem.

In a single‐modal problem, there is a single local optimum that is also the

global optimum.

On the other hand, a multimodal problem may include several local and
global optima.

However, the decision variables that produce a global optimum must all
produce the same value of the global optimum, by definition.
Figure 3 illustrates the surface of one‐dimensional optimization problems
with single‐modal and multimodal decision spaces in which there is one
single optimum.

Different types of decision spaces: (a) maximization problem with single ‐modal surface and one global optimum;
(b) maximization problem with multimodal surface that has one global optimum.
Near‐Optimal Solutions
 A near optimum has a very close but inferior value to the global optimum.

 In some engineering problems, achieving the absolute global optimum is

extremely difficult or sometimes impossible because of the innate complexity of

the problem or the method employed to solve the problem.

 Or achieving the global optimum may be computationally prohibitive.

 In this situation, a near optimum is calculated and reported as an approximation

to the global optimum.

 Near optima are satisfactory in solving many real‐world problems. The proximity

of a near optimum to the global optimum depends on the optimization problem

being solved and the judgment of the analyst.
Near‐Optimal Solutions
Figure 4 depicts the concept of a near optimum in a maximization problem.

Figure 4: Demonstration of near optima in a one‐dimensional maximizing optimization problem.

Simulation
 The process of evaluating the state variables, which are necessary for

estimation of the objective function, and constraints with any decision

variable is known as simulation.

 A simulation model receives the decision variables as inputs and simulates

the system’s state variables.

 Sometimes the simulation model consists of one or more simple

mathematical functions and equations.

 However, most real‐world and engineering problems require simulation

models with complex procedures that most solve systems of equations and
various formulas that approximate physical processes.