Talanta 148 (2016) 641–648

Contents lists available at ScienceDirect

Talanta
journal homepage: www.elsevier.com/locate/talanta

Optimization using the gradient and simplex methods

Víctor Cerdà a,n, Juan Luis Cerdà b, Abubakr M. Idris c
a
Chemistry Department, University of the Balearic Islands, 07122 Palma de Mallorca, Spain
b
Department of Applied Mathematics and Analysis, University of Barcelona, Spain
c
Department of Chemistry, College of Science, King Khalid University, Abha 61321, Saudi Arabia

art ic l e i nf o a b s t r a c t

Article history: Traditionally optimization of analytical methods has been conducted using a univariate method, varying
Received 11 February 2015 each parameter one-by-one holding fixed the remaining. This means in many cases to reach only local
Received in revised form minima and not get the real optimum. Among the various options for multivariate optimization, this
21 May 2015
paper highlights the gradient method, which involves the ability to perform the partial derivatives of a
Accepted 25 May 2015
mathematical model, as well as the simplex method that does not require that condition.
Available online 12 June 2015
The advantages and disadvantages of those two multivariate optimization methods are discussed,
Keywords: indicating when they can be applied and the different forms that have been introduced. Different cases
Multivariate optimization are described on the applications of these methods in analytical chemistry.
Simplex
& 2015 Elsevier B.V. All rights reserved.
Maximum slope
Gradient

1. Introduction minimize for example interferences or an experimental error. In

other cases, targeted values are desirable to be obtained like re-
Optimization is usually a mandatory step when working in covery (100%) in sample treatment and resolution in separation
experimental sciences and engineering, covering from mathema- techniques with maintaining short analysis time.
tical functions and industrial processes to new analytical methods. We are going to focus on how to optimize a minimum. How-
Despite its familiarity and ease of use, univariate optimization ever, if a maximum of a function f(X) is the aim, we just have to
process is time- and reagent-consuming, besides it is unable to minimize the opposite function, i.e. U¼  f(X).
consider interaction effect between conditions. Hence, the max- Although the main aim is to find absolute maxima and minima,
imum efficiency of analytical methods might not be obtained. This mathematical methods frequently provide relative or local max-
because univariate is based on optimizing conditions one-by-one ima and minima depending on the starting point (Fig. 1). There-
by varying levels of one condition while levels of other conditions fore, it is advisable to assure that the maximum or minimum
are held constant. The more optimization effective approach is found are the absolute ones repeating the calculation starting from
multivariate, which optimizes all conditions simultaneously by different initial points.
varying levels of all of them. Accordingly, multivariate optimiza- There are different strategies to obtain the optimum values for
tion could obtain the highest efficiency of analytical methods in different optimization cases, which may be simultaneous (e.g.
the shortest time period. Gradient, Simplex and Evolutionary Operation) [1–3] or sequential
When optimizing a magnitude or a mathematical function, e.g. (e.g. Box–Behnken, Central Composite, Doehlert and Factorial De-
sign) [3]. The calculation method has to be chosen according to
U¼ f(x,y,….,z)¼ f(X), that depends on one or more variables or
each system [1,4]. Below are described some sequential methods
factors, X ¼(x,y,…,z), the function is called the target function and
that can efficiently meet most common optimization cases in ex-
the values of the variables giving an optimum value (maximum or
perimental sciences. The objective function has to be classified in
minimum) for the target function have to be determined. Thus, a
those having derivatives and those not having. Thus, there are two
maximum will be the aim when for example optimizing the effi-
different options,
ciency of a reaction, the maximum signal of an instrument and the
maximum difference between a signal and the instrumental noise.
1. The gradient method is recommended for functions with sev-
On the contrary, a minimum will be desirable if the aim is to eral variables and obtainable partial derivatives, using a varia-
tion of the Newton, Davidon, Fletcher and Powell methods
n
Corresponding author. Fax: þ 34 971173426. 2. For functions with several variables and unobtainable partial
E-mail address: [email protected] (V. Cerdà). derivatives, the simplex method is then the best option.

http://dx.doi.org/10.1016/j.talanta.2015.05.061
0039-9140/& 2015 Elsevier B.V. All rights reserved.
642 V. Cerdà et al. / Talanta 148 (2016) 641–648

Relative
maxima

Absolute
maximum

Fig. 1. Level curves with some local maxima and an absolute maximum.

All cited these are numerical methods, which are based on

refining initial values. These initial values are estimated according
to the knowledge of the particular problem, e.g. when estimating Fig. 3. The gradient method for two variables.
the pK of a functional group, or by values obtained with other
calculation methods of approximation, either graphical or analy- so (Fig. 2b).
tical ones. G(X ) = (fx (X ), …, fz (X )), (1)
If possible, it is advisable to take advantage of algorithms using
derivatives to achieve a better reliability and rapid convergence to In the case of having two variables affecting the target function,
the optimum (gradient method), avoiding direct search despite if we follow the gradient direction, i.e. a way perpendicular to the
their simplicity (simplex method). level curves (Fig. 3), we will be following the curve with maximum
The aim of this review is to provide a general knowledge on the slope. Thus, we will pass from point (Xm) to the next (Xm þ 1)
gradient and simplex methods in the matter of optimization in the considering Eq. (2) for maximizing, and Eq. (3) for minimizing,
experimental sciences, avoiding an excessive use of mathematical being the step “k” always k4 0. It will be clearer if the points Xm,
language, and providing an intuitive justification of the proposed Xm þ 1, etc. are presented in Fig. 4.
methods. More specialized texts in the matter can be found in the
X m + 1 = X m + k G(X m) (2)
literature [5–7]. A good introduction to numerical methods of
analysis can be found in bibliography [8]. Below are described a
X m + 1 = X m –k G(X m) (3)
selection of useful methods to apply in the experimental sciences
field to achieve an optimum.

2.1. Examples of application of the gradient method

2. The gradient method
The gradient method has been widely used in programs to
In the “gradient”, “maximum slope” or “steepest-ascent” refine thermodynamic parameters, such as equilibrium constants,
method, a level set, or a level curve in the case of having two reaction enthalpies and entropies, where the sum of the residuals
variables of U¼ f(X), is a set of points where the studied function squares is minimized (Table 1).
has always the same value. Furthermore, not only U does not vary All programs listed in Table 1 need initial parameters values. As
in this set, but also it is easily demonstrated that the variation of U closer these initial values to the optimum, as faster will be the
respect to the parameters is the maximum in the direction per- convergence to it and the risk to diverge will also be lower.
pendicular to the level set. This direction is determined by the However, the chemical criterion has to be always in mind, other-
gradient vector G(X) (Eq. (1)), whose components are first partial wise absurd results will be obtained, e.g. to obtain three constants
derivatives of function “f”. It has to be taken into account that the of protonation for a diprotic acid because the residuals value is
gradient vector size shouldn’t be constant, otherwise the max- lower.
imum won’t be reached, it will be surpassed (Fig. 2a). The criterion With the last four programs shown in Table 1, two different
to be followed is to reduce the gradient vector size as the slopes do processes can be applied to reach the optimum, i.e. a process

Fig. 2. The gradient method with (a) constant vector size and (b) variable vector size.
 V. Cerdà et al. / Talanta 148 (2016) 641–648 643

Fig. 4. (a) Search method (univariate). (b) Refinement method (multivariate).

Table 1 optimum. Also, difficulties increase along with the number of

Examples of programs using the gradient method. variables (N) to optimize and if the target function cannot be
analytically differentiated. Notably, the optimization of non-dif-
Program Function Parameters to refine Reference
ferentiating functions is quite common in experimental sciences as
MINIQUAD 2 Equilibrium constants [9] in analytical chemistry. An algebraic expression for all variables in
(
U = ∑n pHcalculated − pHexp ) experimental sciences is not usually available.
MINIPOT 2 Equilibrium constants [10]
U = ∑n (Ecalculated − Eexp) When it is difficult or impossible to use the gradient method,
MINISPEF
U = ∑n (Acalculated − Aexp )
2 Equilibrium constants [11] there are great and widely known alternatives provided by direct
and molar sequential procedures. These are generally based on modifications
absorbtivities
MINIPOL 2 Equilibrium constants [12]
of the simplex method. The simplex method was first suggested by
( )
U = ∑n Icalculated − Iexp
and intensities Spendley, Hext and Himsworth [22]. The most common mod-
MINITERM 2 Equilibrium constants [13] ifications used are the Modified Simplex Method (MSM),proposed
U = ∑n (ΔTcalculated − ΔTexp)
and enthalpies by Nelder and Mead [23], and related variants such as the Super
Modified Simplex (SMS) proposed by Routh, Swartz and Denton
based on iterative search following an univariate procedure [24],and the Gaussian adjustment method proposed by Van der
(Fig. 4a), and the other based on multivariate refining exploiting Wiel [25].
the gradient method (Fig. 4b). Below are described in more detail the MSM method and its
With two variables affecting the target function, in the search most representative variants, selected by their usefulness in ex-
method first a wide screening for one variable is made to find a perimental applications.
local minimum for this first variable, maintaining the other vari-
able at a fixed value. Then for the second variable a screening is
made, maintaining the first variable fixed at the value found in the 3. The simplex method
first local minimum, and a second local minimum is found. In
Fig. 4a, it can be seen that an absolute minimum is not achieved The simplex method is based on a geometric figure defined by a
with only two screenings. Univariate screenings have to be re- number of points equal N þ 1 to the number of factors to optimize
peated to go closer to the optimum involving the handling of too (N).
many points. The search method can be helpful as a previous step The simplest scenario comprises two factors and the simplex
to the gradient method application in order to start in a point close would be a triangle. As a way of example, imagine that we aim to
to the optimum, avoiding divergent processes. However, the optimize the combustible flow rate (factor 1, F1) and the oxidizer
number of points is greatly reduced in the gradient method in flow rate (factor 2, F2) to determine an element X using atomic
comparison with the search method. absorption spectrometry.
As a way of example, in the supplementary material can be Fig. 5 shows the level curves with the same response, that is to
found tables of input data and output results of the program say those with absorbances with the same value at different flow
MINITERM when applying first the univariate search method, and rates combination. The aim is to obtain the combination where the
then the multivariate gradient method to refine two equilibrium absorbance is maximal.
constants using enthalpimetric experiments. The optimization starts at the points 1, 2 and 3, which form an
In other applications, the steepest-ascent method was exploi-
ted for optimizing the most effective analytical conditions after
screening using multivariate simultaneous optimization ap-
proaches as full and partial factorial design and Plackett–Burman
[14–20].

2.2. Gradient method difficulties

The gradient method or “steepest descent” and its variants are

considered the best option to evaluate an optimum and its para-
meters or coordinates when it is possible to obtain the derivative
of the target function and a good estimation of the initial values
[21]. However, it is well known that the effectiveness of these
methods decreases if the initial values are far away from the Fig. 5. Simplex advance.
644 V. Cerdà et al. / Talanta 148 (2016) 641–648

equilateral triangle and giving the point 3 as the worst response. Supposing the objective is to find a minimum, for example the
Thus, the response will probably increase in the opposite direction minimum of the sum of squared residuals, the target function “U”
to point 3. Therefore, the reflected triangle 1-2-4 is studied, being will be calculated as in Eq. (6):
4 the reflected point of point 3. A new experiment is carried out to 2
get the absorbance of point 4. If point 2 gives the worst answer of U (PN ) = ∑ (Mcalc − Mexp)
i (6)
the new simplex, the new triangle with the reflected point of 2 is
obtained. This procedure is successively repeated producing a This function is very used to refine experimental parameters
succession of simplexes that move rapidly over the response aiming to obtain a maximum adjustment between the calculated
surface. values of a function Mcalc(PN)i and the experimental data Mexp se-
To achieve a more efficient progress strategy, a series of rules lecting the right N parameters P(1),…..,P(N).
have been proposed as detailed below: In this example, the worst vertex will be that giving the highest
U value. Following the previously described strategy this vertex
1. Rule 1: The new simplex is created discarding the point giving has to be discarded to choose a new one according to a series of
the worst response and replacing it with its mirror image on a simple rules. The method is iterated to progressively improve the
line defined according to the other two points. If the new point U value until the optimum region is located.
gives the following worst response, rule 1 can no longer be The simplex vertexes are numbered, being S(3) the worst, S
applied since the reflected point will be the original one. The (2) the second worst and S(1) the best point. When defining B as
application of rule 1 will result in an endless loop between the the barycenter of the opposite face of S(3), the worst vertex S(3) is
two created simplexes. Thus, rule 2 has to be followed. deleted and the new one “R” is obtained by reflection according to
2. Rule 2: So if the new point gives the worst response, then the Eq. (7).
second worst point has to be selected and its reflected point
will be obtained to create the new simplex. A change in the R = B + (B − S(3)) (7)
progression direction to the optimum is obtained. This happens
The following exceptions should be taken into account:
usually when we are approaching the optimum.
3. Rule 3: Thus, if the new point is close to the optimum, all new
a. If U(R)oU(S(1)) and U(R)4U(D), where D is the expanded, i.e.
points will surpass the response surface apex and next sim-
at double distance of the reflected point, then:
plexes will turn around the optimum. This phenomenon is
shown in Fig. 6 where the turning starts from point 9. D = B + 2(B − S(3)) (8)

b. If U (R) < U (S(3)) and U (R) > US((2)) (9)

Simplex method variants arise with the aim of solving this
difficulty and to speed up the search of the optimum.

c. If U (R) > U (S(3)) (10)

3.1. Simplex method modifications

3.1.1. The modified simplex method

Having N parameters (being N 42), the MSM [23] starts the In case (a) the reflected simplex is expanded and the new
evaluation of the target function “U” in the N þ1 vertexes, e.g. S(1), vertex is D (Fig. 6). On the other hand, in cases (b) and (c) an
S(2) …..S(N þ 1), of the initial simplex. These vertexes can be cal- external and internal contraction is produced and the new
culated from the initial point P(1),…, P(N) and the size of the vertex is C ¼B 70.5(B  S(3)) (Fig. 7).
simplex side “K” as in Eq. (4) The so called critical case is when the worst response is ob-
tained in the new point C. In order to continue it is re-
S(1) P(1) + RP(2) + T…P(N ) + TS(2) P(1) + TP(2) + R…P(N ) commended to follow one of the alternatives described below:
+ TS(N ) P(1) + TP(2) + T…P(N ) + RS(N + 1) P(1) P(2)…P(N ) (4) 1. Massive contraction (Fig. 8), proposed by Nelder and Mead
[23]. S is contracted to S(1) by a factor of 0.5. Then, the
: evaluation of N new vertexes is required before continuing

where,
7
(N + 1)2 + N − 1 (N + 1)1/2 − 1 8
R= K, T = K
2N 2N (5)

S(1)
5

6
C B C R
1
S(3) D

3 4 4

S(2) 2
Fig. 6. Reflected, expanded, and contracted simplex. Fig. 7. Simplex progress with reflections, expansions and contractions.
 V. Cerdà et al. / Talanta 148 (2016) 641–648 645

Fig. 11. Unidirectional progress.

will be the point with the corresponding minimum value of the

quadratic function. Vicinities of B are excluded to avoid the resize
reduction during the process.

3.2. Other modifications

In order to have a faster progress to the optimum and to pre-

vent simplex degeneration due to consecutives deformations
when applying MSM and SMS, i.e. when the simplex is not close to
the optimum region, we have proposed an expanded simplex
translation in the direction S(3) -D (unidirectional progress).
Fig. 8. Simplex massive contraction. When the cases (a) of the MSM takes place, this unidirectional
progress is suggested if U(D) oU(S(1)) and U(Qn)oU(D), where
Q n = B + 2n (D − B) (n = 1, 2)

The last point Q ¼Qn is related to condition U(M) 4U(Q) being

M¼Qn þ i (Fig. 11). Unidirectional progress is recommended only if
n 42; otherwise experience indicates that the expansion of the
MSM (case (a)) is preferable. There are two variants of the uni-
directional progress, namely:

1. Simple unidirectional progress. The new expanded simplex in

(a) is transferred. Hence, the new vertexes are Q¼S(1)þ V, S
(2)þV, …S(N þ1) þV, where V¼Q  S(N þ1).
2. Unidirectional progress with interpolation. The translation
vector is V¼A  D, being A the point giving better response than
Q and P, where P is the point obtained by quadratic inter-
polation respect to values U(D),U(Q), U(M). That is to say P is the
point corresponding to the minimum of the second order
polynomial.
Fig. 9. Simplex translation.

These methods can also be applied in the critical case, pre-

with the algorithm execution. There is the possibility of viously described, with an unidirectional progress of S in the di-
premature convergence in case of error. rection S(3)–S(1). It is enough to use S(1) instead of B to construct
2. Total simplex translation (Fig. 9), proposed by Ernst [7] Q¼Qn (Fig. 12).
3. The simplex is relocated being (1) the barycenter B’ of the Fig. 13 shows the simplex progress witha reflexion, unidirec-
new simplex. Simplex in this case does not contract, but tional translation and internal contraction.
the evaluation of N þ1 requires new vertices (Please check
translation). 3.3. Simplex method and simplex variants applications
4. The variant “next worst”, consists in permuting S(2) with S
(3) when modifying the simplex, to change the simplex As previously stated, the simplex is an optimization method
progress avoiding a premature contraction. based on the logical search strategy of the minimum or maximum
of a function or an experiment with or without mathematical
model. In fact, the significant difference in comparison with other
3.1.2. Supermodified simplex method methods of refining, such as the gradient method, is the no need to
In SMS method (Fig. 10), instead of a sequence of reflections, ex- differentiate a mathematical function.
pansions and contractions to get the new vertexes, the fitting of a As other optimization methods, it requires approximately
knowing the values of the factors to refine, that will be used as
second order polynomial curve for the values of U(S(3)), U(B), U(R)
initial values.
of the vertexes S(3), B and R is used.
As previously stated, one of the greatest problems of refining
The curve is extrapolated further than S(3) and R, a distance
proportional to the distance between S(3) and R. The new vertex

Fig. 10. Supermodified simplex method. Fig. 12. Unidirectional progress of Sin direction S(3)–S(1).
646 V. Cerdà et al. / Talanta 148 (2016) 641–648

three programs can be found in the supplementary material.

4. Scales and finalization criteria

There are common practical considerations for all optimization

methods which are not usually taken into account, despite being
important to assure effectiveness.
On the one hand, a correct election of the scale, i.e. origin and
unit of measurement for all the variables involved, is necessary to
achieve a method sensible to the variation of all factors and to
assure a size adequate to the chosen error bounds. In another
context, finalization criteria have to avoid unnecessary and ex-
cessive iterations and also premature finalization in false optima.

4.1. Scale selection for the variables and the function

A change of scale for the variables involves a correspondence

Fig. 13. Simplex progress with a reflexion, unidirectional translation and internal between the initial variables X¼ (x1,…,xN) of the target function f
contraction.
(X) and the new variables Y ¼(y1,…,yN) obtained changing the
origin and the units of measurement as shown in Eq. (11):
methods is the convergence to a local minimum and not to the
absolute one. Therefore, it is recommended to try different initial Xj = αjY j + βj , (j = 1, …, N ) (11)
points and values for the side size of the initial simplex. Experi-
ence will indicate the most adequate initial values for each parti- Below are listed some basic rules to select the right scale:
cular problem.
1. All involved variables should have a similar magnitude, and of the
3.3.1. Simplex applications order of a unit in the region of interest.
Due to our research needs, we have developed different pro- 2. It is very useful to know a fitted and realistic field of variability of
grams to apply the simplex method to three different cases: each parameter, aj oxj obj, and do the change of scale according
to Eq. (12).
a. Simplex program: to find the minimum of a mathematical Xj = (bj − aj )yj + αj (12)
function, e.g. U¼X2 þY2, where its value is 0 and located in
x¼ 0, and y¼ 0. 3. No change in the scale should be made if it is possible that the
b. SIMFIA program: to find the minimum of an experimental limits aj and bj are wrong.
function knowing its mathematical model (least square ad- 4. For the scale of the function, it is recommended to choose the
justment) [26,27]. values of α and β between  1 and 1, or between 0 and 1, when the
c. ExpSimplex program: to find the minimum in an experimental parameters are in the region of interest.
system not knowing its mathematical model. For example, the 5. The scale should be so that the optimum value of the function
optimization of an atomic absorption instrument adjusting would be 0.Thus, it is better to consider f(x,y) ¼ x2 þ y2 than f(x,
combustible and oxidizer flow rates. The simplex method was y)¼ x2 þ y2 þ 100000.
applied for optimizing the automation of the Winkler metho-
dology using sequential injection analysis (SIA) [28] and opti- In the optimization with derivatives, the scale is so that the
mizing the set-up of dual-laser crossed-beam thermal lens variation of the function is very different respect to the variation of
spectrophotometer [29]. Recently, the Simplex method was the parameters. This issue leads to great differences in the values
applied as well for optimizing four volumetric and two che- of the partial derivatives when changing the variable of derivation.
mical variables controlling the quantification of dissolved Accordingly, scale changes are needed for the variables where αj
oxygen by SIA [28]. Vertex calculation and consequent appli- accomplish Eq. (12). Moreover, it has to be taken into account that
cation including in-line preparation of one reagent were car- the gradient method depends strongly on the selected scale.
ried out in real-time using the software AutoAnalysis.
1 + f (X )
αj =
Actually, examples a and b can be solved with higher efficiency 2f j (X ) (13)
using algorithms including derivatives, which should be preferably
where fj(X) is the partial derivative of “f” respect to Xj.
used to sequential methods.

3.3.2. Simplex method variants applications 4.2. Finalization criteria

The three programs cited previously have been created af-
fording different progress strategies of the simplex during opti- Although an optimization is solved when determining a point X
mization, including translation, unidimensional progress, expan- where G(X) ¼0 or, if knowing the minimum value, f(X) ¼ min, with
sion and interpolation between others.Up to eight variants can be algorithms of finite precision to obtain the convergence to the
chosen, covering from the original of the method by Nelder and optimum (such as those applied in practice), only approximate
Mead [23] to own modifications. The convergence velocity de- solutions will be obtained. That is to say, previous conditions
pends on the kind of problem and the quality of the initial values. should be replaced by others of the kind G(X)¼ 0 or f(X)¼min.
Experience will determine the most convenient variant. Accordingly, by establishing the finalization criteria, the fol-
A more detailed description of applications and lists of these lowing objectives could be achieved:
 V. Cerdà et al. / Talanta 148 (2016) 641–648 647

1. To check the acceptance of an estimated optimum having the 2

U = [1.5 − x(1 − y)]2 + ⎡⎣2.25 − x(1 − y)2⎤⎦
margin of error or the working precision.
2
2. In case of an estimated optimum is not accepted, it is advisable + ⎡⎣2.65 − x(1 − y)3⎤⎦ (15)
to adopt a criterion to stop the process, especially if the process
is slow or requires an elevated number of evaluations of the
2
function, since the method can lead to infinite iterations (
U = 100 y − x2 ) + (1 − x)2 (16)
without convergence to the optimum.
The optimum will be located at (1,1,…, 1) and its value will be
Universal finalization criteria cannot be established for all op- zero. (  1, 2, 1,….,  1, 2, 1) can be taken as initial points [31]
timization methods or cases. However, the correct selection of the
U = (x − y + z )2 + (−x + y + z )2 + (x + y − z )2 (17)
scale should be the norm.
Generally, criteria are elaborated by the user fixing the value of Zangwill [32]
a positive magnitude ε , responsible of the significant figures of the
U = x2 + 2Y 2 + 3Z2 + 4W 2 + (x + y + z + w )4 (18)
function and/or parameters values at the optimum.
To describe some criteria, suppose that the successive refine-
U = (x + 10y)2 + 5(z − t )2 + (y − 2x)4 + 10(z − t )2 (19)
ments of the parameters are X1, X2 ….and the corresponding va-
lues of the function are y1, y2, …. In the case of optimization using With the minimum at (0,0,0,0), (3,  1, 0, 1)is suggested as
derivatives, the criterion of finalization will be to simultaneously initial point.
accomplish the three conditions described below: As in Rosenbrock [31] function, Powell function [33] can be
expanded to 4n variables repeating in groups of 4 variables the
(1) |yk  yk  1| o ε(1 þ|yk|) above expression.
(2) I|xjk  xjk  1| o(ε(1 þ |xjk|))½ (j¼1,…,N) 2
(3) |fj(Xk)| r ε⅓(1 þ|yk|) U=
n
∑i = 1 (n − ∑ n
j=1
( ) )
(cos xj + i 1 − cosxj − senxj )
(20)
where Xk ¼(xjk)¼ (x1,…,XN
k ). Trigonometric function, with n variables for X¼ (x1, x2, …., xn).
Condition (2) can be ignored if condition (1) is accomplished (1/n, 1/n,…, 1/n) will be the initial point.
and a right selection of scales has been made. In order to study the different optimization methods we have
In the simplex method it can be ended if f(S(3))  f(S(1)) o ε (1 conducted a great number of assays with a random selection of the
þf(S(1)); or if knowing the minimum value OPT of the function f, if initial point and of the size K of the first simplex.
f(S(1))–OPTo ε. We have mainly focused in comparing the Nelder and Mead
Given the possibility of an experimental error, it is re- method [23] and the two variants of the unidirectional progress,
commended that an extra condition, described below, should be together with the different options in the critical cases, i.e. massive
applied in order to avoid a premature finalization. contraction, translation and unidirectional progress. The number
of evaluations needed to reach the optimum has been determined.
( )
|x3j − x1j | < (ε 1 + |x1j | )½ (j = 1, … ... , N ),
For all functions, the unidirectional progress approximation has
where X3 ¼S(3) and X1 ¼ S(1). proved to be more efficient than just the expansion of the reflected
Nelder and Mead [23] adopt the finalization criterion to ac- simplex in MSM. This advantage increases when increasing N, with
the relation of the distance of the initial point and the optimum
complish the following condition.
and also with the size K of the first simplex.
⎛ N+1 ⎞1/2 Nonetheless, in the critical cases, the massive contraction ap-
1 ⎜ 2⎟
∑ [f (B ) − f (S (i ))] ⎟ <ε
N + 1 ⎜⎝ i = 1
proach is slightly more beneficial than the unidirectional progress
⎠ and the translation when the unidirectional progress has been
where S(1),…,S(N þ 1)are the N þ1 vertexes of the last simplex and used instead of the expansion. This can be attributed to the fact
B the barycenter of the opposite face to S(3). that this progress avoids premature contraction or distortion of
When dealing with experimental functions and taking into the simplex.
account possible measurement errors, it is advisable to apply the In the unidirectional progress there is no improvement when
rule “N þ1” since the value of the function is reevaluated in S(1) if using the quadratic interpolation. Thus, results are similar and in
this vertex gives the best response after N þ1 consecutive general, the simple unidirectional progress is preferable.
Therefore it is advisable to use the MSM, with massive con-
iterations.
traction in the critical cases, improved with a simple unidirectional
progress in the cases (a) when U(D)4 U(R) 4U(S(1))and U(Q2) 4U
(Q1)oU(D). This modification of the sequential Simplex method is
5. Assay functions very simple and effective. As in the MSM, no especial mathema-
tical knowledge is required and the calculations involved are tri-
Once elaborated an optimization computerized program, it has vial and easy to program.
to be evaluated and compared with others.
It is advisable to check its operation by stages. First with simple 5.1. Robustness of the methods
problems and then with more complex ones, increasing the
number of variables and the precision required to finalize. Then if Finally, optimization shouldn’t be end when obtaining the op-
the components work properly, the program has to be checked timum parameters. Results should be complemented, i.e. evalu-
with different functions [30]. ating each parameter behavior taking its optimum as the central
Some of the most assayed functions to check the efficacy of point of a new assay. Thereafter, the robustness of the method
different alternatives are: should be evaluated. The method will be more robust when
showing higher tolerance to variations of the studied parameters
U = x12 + … … + x n2 (from N = 2 up to 8) (14) in the region of the found optimum.
648 V. Cerdà et al. / Talanta 148 (2016) 641–648

5.2. Flow analysis applications References

In 2010, an overview on the applications of the SIMPLEX al- [1] P. Gans, Coord. Chem. Rev. 19 (1976) 99–124.
gorithm for the optimization of analytical systems based on un- [2] B. Dejaegher, Y.V. Heyden, Sequential optimization methods, Ref. Modul.
Chem. Mol. Sci. Chem. Eng. Compr. Chemom. 1 (2009) 547–575.
segmented flow techniques was published by our group [3] L.V. Candioti, M.M. De Zan, M.S. Cámara, H.C. Goicoechea, Experimental design
[28,30,34]. The different response functions used for evaluation of and multiple response optimization. Using the desirability function in analy-
the optimization experimental data were compared and their ad- tical methods development, Talanta 124 (2014) 123–138.
[4] L.A. Sarabia, M.C. Ortiz, Response surface methodology, Ref. Modul. Chem.
vantages and shortcomings and the problematic of using SIMPLEX Mol. Sci. Chem. Eng. Compr. Chemom. 1 (2009) 345–390.
with a special focus on real-time application was shortly dis- [5] J.E. Dennis Jr., R.B. Schnabel, Numerical Methods for Unconstrained Optimi-
zation aAnd Nonlinear Equations, Prentice-Hall, 1983.
cussed. The analytical applications and parameters optimized by
[6] R. FIetcher, Practical Methods of Optimization, Vol I, Wiley, 1980.
SIMPLEX method were summarized in tabular form. [7] D.M. Himmelblau, Applied Nonlinear Programing, McGraw, New York, 1972.
[8] A.M. Cohén, Análisis Numérico, Reverte, Barcelona, 1977.
[9] P. Gans, A. Sabatini, A. Vacca, Talanta 43 (1996) 1739–1753.
[10] F. Gaizer, A. Puskás, Talanta 28 (1981) 565.
6. Conclusions [11] F. Gaizer, A. Puskás, Talanta 28 (1981) 925.
[12] V. Cerdà, R. Jara, Thermochim. Acta 87 (1985) 13.
[13] R. Forteza, V. Cerdá, Talanta 32 (1985) 1159.
1. In sequential optimization methods, the gradient method is [14] H. Chen, J. Zhang, Y. Dang, G. Shu, J. Chem. Pharm. Res. 6 (2014) 612–616.
recommended for functions with several variables and ob- [15] L. Escuder-Gilabert, Y. Martín-Biosca, S. Sagrado, M.J. Medina-Hernández, J.
Chromatogr. A 1363 (2014) 331–337.
tainable partial derivatives while the simplex method is re-
[16] G. Di Nicola, M. Pacetti, F. Polonara, G. Santori, R. Stryjek, J. Chromatogr. A 1190
commended for functions with several variables and un- (2008) 120–126.
obtainable partial derivatives. [17] K. Selber, F. Nellen, B. Steffen, J. Thömmes, M.-R. Kula, J. Chromatogr. B 743
(2000) 21–30.
2. It is advisable to start in as close to the optimum as possible.
[18] A. Herrero, M.C. Ortiz, M.J. Arcos, J. López-Palacios, Analyst 119 (1994)
Methods of approximate calculation can be used to fulfill this 1585–1592.
purpose. [19] P. Petersson, N. Lundell, K.E. Markides, J. Chromatogr. A 623 (1992) 129–137.
3. The gradient method is the best option when possible. [20] D. Wienke, C. Lucasius, G. Kateman, Anal. Chim. Acta 265 (1992) 211–225.
[21] J. Cerdá, V. Cerdá, Cuadernos de Ciencias y Técnicas Ambientales, in:
4. Optimization has to be repeated starting from different initial M. Blanco, V. Cerdà (Eds.), Serie Química Analítica, AEST, Barcelona, 1988.
points to assure that the optimum found is the absolute max- [22] W. Spendley, G.R. Hext, F.R. Himsworth, Technometrics 4 (1962) 441.
imum or minimum, and not a local one. [23] J.A. Nelder, R. Mead, Comput. J. 7 (1965) 308.
[24] M.W. Routh, P.A. Swartz, M.B. Denton, Anal. Chem. 49 (1977) 1422–1428.
5. Various programs have been developed to refine thermo- [25] P.F.A. van der Wiel, Anal. Chim. Acta 122 (1980) 421–433.
dynamic parameters based on the gradient method. [26] M. Blanco, V. Cerdà, J. Coello, J. Gené, H. Iturriaga, S. Maspoch, Anal. Lett. 25
(1992) 543–560.
[27] V. Cerdà, J.L. Cerdà, Optimization of analytical techniques using the gradient
and simplex methods. 19th International Conference on Flow Injection Ana-
lysis and Related Techniques. Fukuoka, Japan,.
[28] B. Horstkotte, A.T. Sánchez, C.M. Duarte, V. Cerdà, Anal. Chim. Acta 658 (2010)
Acknowledgments
147.
[29] A. Cladera, C. Tomás, J.M. Estela, V. Cerdà, G. Ramis, Anal. Chim. Acta 282
The authors acknowledge financial support from Spanish (1993) 613–623.
[30] M. Blanco, V. Cerdà (Eds.), Temas avanzados de Quimiometría, University of
Ministry of Economy and Competitiveness (MINECO) through the Balearic Islands, Palma de Mallorca. Spain, 2007, ISBN: 987-84-8384-006-
Project CTQ2013-47461-R co-financed by FEDER funds. 1.
[31] H.H. Rosenbrock, Comput. J. 3 (1960) 175.
[32] W.I. Zangwill, Manag. Sci. 13 (1967) 334.
[33] M.J.D. Powell, Non convex minimization calculations and the conjugate gra-
Appendix A. Suplementary Information dient method, in: Lecture Notes in Mathematics, vol. 1066, Springer (1984), p.
122–141.
[34] Burkhart Horstkotte, Carlos M. Duarte, Víctor Cerdà, TrAC Trends Anal. Chem.
Supplementary data associated with this article can be found in the 29 (2010) 1224–1235.
online version at http://dx.doi.org/10.1016/j.talanta.2015.05.061.