4962 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 38, NO.

5, SEPTEMBER 2023

A Modal-Based Initial Estimate for the Newton Solution

of Ill-Conditioned Large-Scale Power Flow Problems
Laice Neves de Oliveira , Francisco Damasceno Freitas , Senior Member, IEEE,
and Nelson Martins , Life Fellow, IEEE

Abstract—Based on a modal approach, this letter identifies the Several publications have dealt with the ill-conditioning prob-
primary cause for the ill-conditioning of a Power Flow Problem lem proposing different techniques [5], [7], [8]. In [9], the
(PFP) and proposes a method to circumvent this numerical weak- ill-conditioning of the PF problem was attenuated by introduc-
ness. The technique consists in modifying the ill-conditioned PFP
Jacobian matrix for the standard Newton method initial iterate ing empirically a uniform perturbation to the diagonal of the
by moving away just its smallest magnitude eigenvalue from near Jacobian matrix to the first iteration of the Newton solver.
zero. The state deviations for this modified condition is then effi- This letter demonstrates that the primary cause of the ill-
ciently computed. This procedure is performed by adding a 1-rank conditioning problem is associated with the smallest magnitude
perturbation matrix to the Jacobian matrix, but just for the first eigenvalue of the PF Jacobian matrix’s first iterate. In addition,
iteration and removing it for all others. Experiments performed in
two ill-conditioned large-scale systems demonstrate the method’s it is proposed a procedure to circumvent this problem by shift-
efficiency. ing away from zero the smallest magnitude eigenvalue of the
Jacobian matrix. For this purpose, the smallest eigenvalue and
Index Terms—Power flow, Newton method, ill-conditioned
system, Sherman-Morrison formula, modal analysis.
its respective right- and left-eigenvectors are computed. Then, a
1-rank perturbation matrix formed with these partial eigendata is
added to the Jacobian matrix. Finally, an approximation for the
I. INTRODUCTION deviation of states is efficiently computed, considering the same
HE Power Flow (PF) is the most pervasive computational mismatches determined for the first iteration. Experiments con-
T tool in power system analysis, and Newton’s iterative
algorithm is its most efficient engine. As an iterative solver,
ducted on two ill-conditioned large-scale power system models
demonstrate the proposed technique’s efficacy.
the Newton method depends on an initial estimate, preferably
assigned near the solution. However, it turns out that some II. THE POWER FLOW PROBLEM FORMULATION
factors, such as the proliferation and use of renewable energy
sources [1], frequent reversal of flows [2], and the load variation The PF solution consists in solving a well-known set of
itself can lead to significant changes in the operating point. As a nonlinear equations denominated Power Balance Equations
result, the voltage magnitude and angle profile of the system can (PBEs) [3] that can be expressed as a vector of functions:
be considerably altered. These aspects can introduce difficulties
g(x) = 0, (1)
in selecting an initial estimate to the PF solver.
A usual initializing strategy for the PF solver is to adopt a where x is the state vector defined as x = [θ T V T ]T ∈
flat start type estimation [3]. This initial estimate may, however, Rn ; g(x) = [gPT gQ T T
] , in which gP k , k = 1, . . . , n − 1, and
be far from the solution of interest in the studied power system gQk , k = 1, . . . , NP Q , for Nb buses are
model. As a result, the straightforward utilization of the classical
Newton method may diverge, even though the problem has a Nb

solution. For failure of convergence, the system is said to be gP k = Vk Vm [Gkm cos(θkm ) + Bkm sin(θkm )] − Pksp ,
ill-conditioned [4], [5], although the ill-conditioning is related m=1
to the perturbation of a linear system [6].
Nb

Vm [Gkm sin(θkm ) − Bkm cos(θkm )] − Qsp
Manuscript received 27 January 2023; revised 15 May 2023 and 3 July 2023;
accepted 7 July 2023. Date of publication 13 July 2023; date of current version gQk = Vk k ,
21 August 2023. This work was supported by University of Brasilia (UnB) by the m=1
Graduate Program in Electrical Engineering (PPGEE). Paper no. PESL-00035-
2023. (Corresponding author: Francisco Damasceno Freitas.) being n = 2NP Q + NP V (NP Q and NP V are the number of
Laice Neves de Oliveira and Francisco Damasceno Freitas are with the Univer- PQ and PV buses, respectively). Vk and θk are, respectively, the
sity of Brasilia, Brasilia 70910-900, Brazil (e-mail: [email protected];
[email protected]). voltage magnitude and phase angle of a nodal voltage phasor,
Nelson Martins is with the National Academy of Engineering of Brazil, Rio V k = Vk ∠θk ; θkm = θk − θm ; Pksp is the active power injected
de Janeiro 20040-001, Brazil (e-mail: [email protected]). into bus #k, while Qsp
k is the reactive power injected into the same
Color versions of one or more figures in this article are available at
https://doi.org/10.1109/TPWRS.2023.3295178. bus; Gkm and Bkm are the real and imaginary parts of the entry
Digital Object Identifier 10.1109/TPWRS.2023.3295178 Y km = Gkm + jBkm of the system bus admittance matrix.

0885-8950 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

uthorized licensed use limited to: Indian Institute Of Technology (Banaras Hindu University) Varanasi. Downloaded on September 17,2023 at 21:50:32 UTC from IEEE Xplore. Restrictions apply
 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 38, NO. 5, SEPTEMBER 2023 4963

The ith iteration of the Newton method to iteratively solve (1)

is given by
x(i+1) = x(i) + Δx(i) , (2)
where Δx(i) is calculated for a mismatch g(x(i) ) and Jacobian
matrix J(x(i) ) = ∂g(x)
∂x |x(i) in such a way that
  −1  
Δx(i) = − J x(i) g x(i) . (3)

The value x(i+1) converges numerically to the root x∗ when

||g(x∗ )||∞ <  for  > 0 sufficiently small.
Fig. 1. Plot illustrating the participation factors associated to the SEJ.
III. MODAL-BASED JACOBIAN MATRIX PERTURBATION
The standard Newton method may fail when attempting to
solve an ill-conditioned system in the context of the problem needed for the initial iterate, as done for the classical Newton
defined in [4]. In order to mitigate the ill-conditioning effect solver, and that the sparsity of the standard PF is preserved.
of the PF problem in the Newton’s method first iteration, a Besides, only one additional forward and backward substitution
modal-based technique is proposed to move the smallest mag- operation is performed to determine Δu1 . Finally, the remainder
nitude eigenvalue of the Jacobian (SEJ) matrix from near zero iterates should be determined according to (2) as for the standard
(see [10] and references therein for additional information on Newton solver, without using the 1-rank perturbation matrix
the eigenvalue application to the PF problem). anymore. In case of λ1 be complex, replace u1 in (4) by [u1 u∗1 ];
Let λ1 be the real SEJ and u1 and v1 its respective real right- v1 by [v1 v1∗ ]; and Λ, by α · diag{λ1 ; λ∗1 }, where z ∗ is the
and left-eigenvector. Then a perturbed Jacobian matrix Ĵ = notation for the complex conjugate of z. The perturbation for
J + αu1 λ1 v1T has the original SEJ changed to λ̂1 = λ1 + αλ1 . this situation would be a 2-rank type.
When α > 1, and assuming λ1 > 0, the magnitude of λ̂1 grows.
Then, α can be selected to yield |λ2 | < αλ1 , where λ2 is the
second SEJ. In the cases we analysed the smaller eigenvalues IV. CASE STUDIES
were real positive numbers, and λ2 was relatively away from In this section, we describe experiments carried out on two
zero and is not so dominant as λ1 . Therefore, it is expected that large-scale system models in order to demonstrate the perfor-
the ill-conditioning related to the initial iterate for determining mance of the proposed modal-based perturbation technique to
the approximate increment Δx for (2) is significantly reduced. the PF problem. The models include a 13659- and a 70k-bus
For the initial iteration, given x(0) , the classical form to systems. All simulations are performed considering an adapted
compute the increment Δx(0) is by solving the linear sys- code from the Matlab-based MATPOWER tool [12] running on
tem J(x(0) )Δx(0) = −g(x(0) ) and so to obtain x(1) = x(0) + an Intel Core i7-5500 U with 16 GB of RAM; 2.40 GHz; 64-bit
Δx(0) . However, when J(x(0) ) is ill-posed, mainly as a result of operating system.
the poor initialization, we propose to introduce a perturbation The simulations consisted in computing the smallest mag-
term on the Jacobian matrix in such a way as to move just the nitude eigenvalues of the Jacobian matrix J(x(0) ) in (4) that
smallest magnitude eigenvalue of J(x(0) ). is responsible for its ill-conditioning and introducing a 1-rank
Using the 1-rank perturbation matrix for J(x(0) ), we obtain perturbation on the Jacobian matrix. The smallest eigenvalue
an approximate deviation for the initial iterate Δx̂(0) as λ1 of J(x(0) ) and its associated right- and left-eigenvectors
      were calculated using the eigs code available in Matlab’s
J x(0) + u1 Λv1T Δx̂(0) = −g x(0) (4)
kernel. Similar computation was performed for the two other
where Λ = αλ1 ; α > 1 must be fixed in order to move away smallest magnitude eigenvalues that are immediately larger
from near zero the SEJ (deflation action); and the eigenvectors than λ1 . The three SEJs for the 70k-bus system load-base are
u1 and v1 are dense real vector. λ1 = 3.98 × 10−4 , λ2 = 2.72 × 10−3 and λ3 = 6.69 × 10−3 .
An efficient way for solving (4) is applying the 1-rank For the 13,659-bus system, they are λ1 = 4.84 × 10−4 , λ2 =
Sherman-Morrison inverse technique [11] for the solution 7.56 × 10−3 and λ3 = 2.26 × 10−2 . When α = 100 is used for
   −1   the 1-rank perturbation in the 70 k system, the SEJ will be λ2 .
Δx̂(0) = − J x(0) + u1 Λv1T g x(0) With this 1-rank perturbation, the new SEJ becomes not anymore
critical to the point of contributing to the ill-conditioning of the
 −1 T
= Δx(0) − Δu1 Λ−1 + v1T Δu1 v1 Δx(0) , (5) resultant linear system. The participation factors [3] associated
with the SEJ were determined and are depicted in Fig. 1. The first
and Δx(0) = −[J(x(0) )]−1 g(x(0) ), Δu1 = [J(x(0) )]−1 u1 . half of the plot from 1 to 69,999 comprises angle states, while
The expressions (5) gives the modified increment for the from 70 k to 134,104 is associated with voltage magnitudes. The
state vector for the first iterate, allowing the computation of participation factors are normalized in relation to the maximum
x(1) ≈ x(0) + Δx̂(0) . Note that just one LU factorization is absolute variable of each one of these two regions. This double

uthorized licensed use limited to: Indian Institute Of Technology (Banaras Hindu University) Varanasi. Downloaded on September 17,2023 at 21:50:32 UTC from IEEE Xplore. Restrictions apply
 4964 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 38, NO. 5, SEPTEMBER 2023

normalization was necessary since the participations between TABLE I

ITERATIONS REQUIRED FOR THE BASE CASES OF THE TWO SYSTEMS
the two set of variables, the set of voltage magnitude states are
much smaller.
The participation factors for λ1 presents a high participation
in a large number of phase angle states. When we quantify the
participation with more than 90%, the mode λ1 has 7.2% of states
from a total of 134,104, but only 1 voltage magnitude state is in
this range. The λ2 and λ3 have significant participation in only
0.083% and 0.010% states, respectively and their participation
factor contributions are again dominant for states characterized
by phase angles. Therefore, the most critical eigenvalue for the
ill-conditioning of the first iteration Jacobian is λ1 . Then, this
modal information was explored to solve the PF problem. The
1-rank perturbation matrix for J(x(0) ) was applied. Also, the
shift-δ approach was employed for the sake of comparison. For
this aim, a uniform perturbation to the diagonal of the Jacobian
matrix was applied by introducing a scaling factor δ > 0. Exper-
iments were performed considering the classical Newton (CN),
its fast decoupled (CFD) and the Heun-King-Werner (HKW) [8],
[9] solvers. In this case, the perturbed Jacobian matrix was
replaced in (4) by J(x(0) ) + δI, as proposed in [9], where δ
uniformly shifts all eigenvalues of J(x(0) ).
The value δ was assigned such that δ > λ1 , but in [9] this
factor is assigned empirically. This means that the selection of
δ indirectly depends on the knowledge of λ1 . Note that the
selection by trial-and-error can be more expensive than the
Fig. 2. Convergence characteristics of the CN method - case_ACTIVSg70 k.
computation of a lower threshold as the smallest magnitude
eigenvalue.
The simulations with the 1-rank modal-based approach were TABLE II
carried out for α = 100, i.e., λ1 was moved to λ1 ≈ 3.98 × CPU TIME IN SECONDS FOR THE CASES IN TABLE I
10−2 . Three values for δ, 0.001, 0.01 and 0.1 were adopted for
the shift-δ. Therefore, the computation of λ1 is applied for both
1-rank modal-based and shift-δ methodologies. A tolerance of
10−8 was adopted for convergence of the power mismatches in
pu, and a limit of 20 iterations was used.
All simulations are performed with the flat start estimate and
the perturbation term is introduced just for the initial iteration of
a solver. Both case13659pegase and case_ACTIVSg70 k diverge
when the CN or the CFD solvers are used without the 1-rank
perturbation term added to the Jacobian matrix. The HKW approach is sensitive to a set of parameters and in the simulations
solver needs a set of parameters which were suggested in [8] was used a set of universal values as presented in [9].
as universal parameters. They were used in [9] for the shift-δ The results indicate the benefit of knowing λ1 for all method-
approach and were also here adopted in the simulations for the ologies investigated, because it is a reference to set α as well
1-rank solver. as δ for shifting. For example, δ can be set by using the
Table I shows, in the 2nd and 3 rd columns, the iterations bisection method, starting from a minimum value δmin > λ1
required for convergence of the simulated cases to the perturbed and a maximum value δmax >> λ1 . Authors’ experience has
situations for the case13659pegase and case_ACTIVSg70 k, demonstrated that only some additional PFP computations are
respectively. For the CFD, the table indications refer to the needed to implement this search. Fig. 2 depicts plots illustrating
P- and Q-iteration loops, respectively. When using the 1-rank the convergence characteristics for the CN solver profile in
perturbation matrix for J(x(0) ), the three solvers obtained the Table I for the 70k-bus system.
PF problem solution for all systems. The shift-δ approach for Table II shows the computational costs demanded by each
the CN demonstrated to be equivalent to the 1-rank perturbation method with its respective perturbation approach in J(x(0) ),
for δ = 0.01. However, for other shift-δ values it has a poorer according to the simulations listed in Table I. The mean CPU
performance. A similar conclusion was verified for the CFD time for calculating only one eigenvalue and its right- and left-
approach. On the other hand, the worst performance was verified eigenvectors is 0.2877 s and 2.1545 s for the case13659pegase
for the HKW solver which required a number of iterations higher and case_ACTIVSg70 k, respectively, and they were excluded
than the CN solver. This fact is justified because the HKW from CPU times shown in the table. It is observed that the

uthorized licensed use limited to: Indian Institute Of Technology (Banaras Hindu University) Varanasi. Downloaded on September 17,2023 at 21:50:32 UTC from IEEE Xplore. Restrictions apply
 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 38, NO. 5, SEPTEMBER 2023 4965

CN method (1-rank) converged in a smaller CPU time than the “diagonal shift” has already gained practical use [9], we suggest
CN method with the shift-δ approach. A similar performance that power flow developers implement this strategy and the
was verified for the CFD approach. Furthermore, the CFD has “modal-based shift” one, so that the engineer may decide on
presented the best performance for the solver requiring approxi- which one to use, following tests on his own system model.
mately only 1/3 of the computational cost of the CN. The results The work suggests that the modal 1-rank perturbation ap-
for the HKW solver were not shown, since they had the worst proach here applied to the first iteration of the PF problem may
performance even when compared with the CN. The results are be a promising research field for solving the ill-conditioning of
important, not only because of the convergence of the Newton practical nonlinear problems in other fields of engineering.
method for ill-conditioned large-scale systems but also because
of the excellent performance in terms of computational time and REFERENCES
accuracy achieved by the CFD solver.
[1] Z. X. Weng, L. B. Shi, Z. Xu, L. Z. Yao, Y. X. Ni, and M. Bazargan,
Simulations were also carried out considering the reactive “Effects of wind power variability and intermittency on power flow,” in
limits in power generation. It was used the classical strategy for Proc. IEEE Power Energy Soc. Gen. Meeting, 2012, pp. 1–7.
computation of the PF with the reactive limit as defined in the [2] G. N. Taranto et al., “Power flow control for an embedded HVDC link to
integrate renewable energy in Brazil,” Electric Power Syst. Res., vol. 211,
tool MATPOWER [12]. First, the PF base case is solved with free 2022, Art. no. 108504.
limits and estimate flat start to initialize the computation. Then, [3] P. Kundur, Power System Stability and Control (EPRI, Power Engineering
the 1-rank modal approach is applied to allow the convergence of Series), 1st ed. New York, NY, USA: McGraw-Hill, 1994.
[4] F. Milano, “Continuous Newton’s method for power flow analysis,” IEEE
the PF problem. From the subsequent iteration, now including Trans. Power Syst., vol. 24, no. 1, pp. 50–57, Feb. 2009.
the limits, the result of the previous iteration was adopted as [5] F. Milano, “Implicit continuous Newton method for power flow analysis,”
an initial estimate. Experiments in both case13659pegase and IEEE Trans. Power Syst., vol. 34, no. 4, pp. 3309–3311, Jul. 2019.
[6] G. H. Golub and C. F. V. Loan, Matrix Computations. Baltimore, MD,
case_ACTIVSg70 k, assuming their limits according to the data
USA: The Johns Hopkins Univ. Press, 1996.
bank from [12], converged after two and five limit iterations, [7] S. Iwamoto and Y. Tamura, “A load flow calculation method for ill-
respectively. conditioned power systems,” IEEE Trans. Power App. Syst., vol. PAS-100,
no. 4, pp. 1736–1743, Apr. 1981.
[8] M. Tostado-Véliz, S. Kamel, and F. Jurado, “An efficient power flow
V. CONCLUSION approach based on Heun and King-Werner’s methods for solving both
well and ill-conditioned cases,” Int. J. Elect. Power Energy Syst., vol. 119,
This letter describes a new method for detecting the pri- 2020, Art. no. 105869.
mary cause of ill-conditioning in a large-scale PF problem and [9] F. D. Freitas and L. N. d. Oliveira, “Conditioning step on the initial estimate
provides an efficient technique for finding its solution. With when solving ill-conditioned power flow problems,” Int. J. Elect. Power
Energy Syst., vol. 146, 2023, Art. no. 108772.
this objective, the smallest magnitude eigenvalue and respective [10] T. Ding, C. Li, Y. Yang, R. Bo, and F. Blaabjerg, “Negative reactance
right- and left-eigenvectors of the Jacobian matrix of the initial impacts on the eigenvalues of the Jacobian matrix in power flow and type-1
iterate of the PF must be computed, and a 1-rank perturbation low-voltage power-flow solutions,” IEEE Trans. Power Syst., vol. 32, no. 5,
pp. 3471–3480, Sep. 2017.
matrix is added to the Jacobian matrix. The states calculated this [11] W. W. Hager, “Updating the inverse of a matrix,” SIAM Rev., vol. 31, no. 2,
way allowed us to ensure adequate convergence for the classical pp. 221–239, 1989.
Newton method. Experiments conducted on two ill-conditioned [12] R. D. Zimmerman, C. E. Murillo-Sanchez, and R. J. Thomas, “MAT-
POWER: Steady-state operations, planning and analysis tools for power
large-scale systems demonstrated the robust performance of systems research and education,” IEEE Trans. Power Syst., vol. 26, no. 1,
the modal-based technique applied to the PF problem. As the pp. 12–19, Feb. 2011.

uthorized licensed use limited to: Indian Institute Of Technology (Banaras Hindu University) Varanasi. Downloaded on September 17,2023 at 21:50:32 UTC from IEEE Xplore. Restrictions apply