4956 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO.

6, JUNE 2017

Nash Equilibrium Topology of Multi-Agent

Systems With Competitive Groups
Jingying Ma, Yuanshi Zheng, and Long Wang

Abstract—Competition is ubiquitous in nature. This pa- systems pertain to consensus problem [1], [8], [9], formation
per studies competition phenomena of multi-agent systems control problem [4], containment control problem [10], [11],
consisting of three groups of agents. In order to achieve controllability analysis [12], [13], flocking [14], and so on.
maximal influence, the first and the second groups send
information to the third group, which leads to competition. In multi-agent systems, each agent is an individual who makes
First, we formulate this competition as a noncooperative decision independently. When agents have the same interest,
game in which the first and the second groups are two play- agents will cooperate with their local neighbors by sharing in-
ers. Players decide agents who send and receive informa- formation. Consensus is a fundamental cooperative behavior,
tion. Consequently, the interaction topology of the system
which means that a team of agents agree on a common goal
is generated from players’ strategies. Therefore, we define
the interaction topology decided by Nash equilibrium of the of interest, e.g., the incremental cost in power generations, the
game as the equilibrium topology of the system. Second, heading of a UAV formation, or the target position of a robotic
the necessary condition is established for equilibrium topol- team. In [1], a simple model was introduced to study the behav-
ogy. For the case that the third group’s interaction graph is ior of consensus. Subsequently, some theoretical explanations
a tree or has a center vertex, interchangeable Nash equilib-
were provided by using graph theory [8]. Cooperative game
rium solutions are obtained. Moreover, due to competition,
the agents of the third group might reach consensus under theory is also utilized to ensure consensus reaching [15]. Up to
the equilibrium topology. Finally, when the third group’s in- now, numerous results on consensus were reported, to name but
teraction graph is bidirected, the necessary and sufficient a few, consensus with switching topologies [16], finite-time con-
condition is given for the equilibrium topology. The equi- sensus [17], [18], optimal consensus problem [19]–[21], group
librium topology is also presented for the scenario where
consensus [22], consensus for heterogeneous multi-agent sys-
the third group’s interaction graph is a bidirected circulant
graph. tems [23], and for switched multi-agent systems [24], [25]. In
[6], a consensus protocol was proposed for smart-grid comput-
Index Terms—Equilibrium topology, multi-agent systems, ing. By selecting the incremental cost of each generation unit
noncooperative game.
as the consensus variable, Zhang and Chow [7] gave an incre-
I. INTRODUCTION mental cost consensus algorithm to solve economic dispatch
problem in a distributed manner. As an extension of consensus,
N THE last decade, distributed coordination and coopera-
I tive control of multi-agent systems (MASs) have captured
tremendous attention from a wide range of academic disciplines,
containment control of multi-agent systems has also been widely
studied recently. Some researchers considered this problem un-
der a leader-based framework. Ji et al. [26] proposed a hybrid
such as biology, engineering, social science, etc. [1]–[3]. This is stop-go strategy to achieve containment with fixed topology.
mainly due to their diverse applications, such as tracking control Notarstefano et al. [27] investigated containment control of first-
of robotic teams [2], unmanned air vehicles (UAVs) formations order MASs with switching topologies. Other researchers inves-
[4], distributed power dispatch, and distributed energy manage- tigated containment control problem by classifying the agents
ment of smart grids [5]–[7]. Fundamental aspects of multi-agent into boundary agents and internal agents. Liu et al. [10] showed
Manuscript received August 14, 2016; revised December 11, 2016; that the states of internal agents converge to a convex combina-
accepted January 29, 2017. Date of publication March 1, 2017; date of tion of the boundary agents with weakly connected topologies.
current version May 10, 2017. This work was supported in part by the What will happen when agents have different interests? This
NSFC under Grant 61533001, Grant 61375120, Grant 61304160, and
Grant 61563043 and in part by the Fundamental Research Funds for may produce noncooperation behaviors. For examples, price ne-
the Central Universities under Grant JB160419, and in part by the Young gotiation in a smart grid consisting of electric power companies
Talent Fund of the University Association for Science and Technology and their customers and competition of two political parties,
in Shaanxi Province of China under Grant 20160208. (Corresponding
author: Long Wang.) which run for election in social networks. Some methodologies,
J. Ma and Y. Zheng are with the Center for Complex Systems, School such as noncooperation game theory, signed graph theory, and
of Mechano-Electronic Engineering, Xidian University, Xi’an 710071, optimization theory might be applied to analyze their behaviors.
China (e-mail: [email protected]; [email protected]).
L. Wang is with the Center for Systems and Control, College of Engi- By virtue of game theory, Mohsenian-Rad et al. [28] presented
neering, Peking University, Beijing 100871, China (e-mail: longwang@ a distributed demand-side energy management system for smart
pku.edu.cn). grids with digital communication infrastructure. Gu [29] em-
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. ployed differential game theory to achieve formation control.
Digital Object Identifier 10.1109/TIE.2017.2674625 The notion of graphic game was introduced in [30]. Gharesifard

0278-0046 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
MA et al.: NASH EQUILIBRIUM TOPOLOGY OF MULTI-AGENT SYSTEMS WITH COMPETITIVE GROUPS 4957

and Cortés [31] considered a zero-sum game for two networks Notation: Throughout this paper, we denote the set of real
engaged in a strategic scenario. Clark and Poovendran [32] for- numbers by R, the set of n × m real matrices by Rn ×m . Denote
mulated the problem of maximizing influence in social networks by 1n (or 0n ) the column vector with all entries equal to one
with competitive ideas as a Stackelberg game. In [33], signed (or all zeros). In denotes an n-dimensional identity matrix. For
graph theory was employed to consider group synchronization a column vector b = [b1 , b2 , . . . , bn ]T , diag{b} is a diagonal
problem for multi-agent systems with competitive interactions. matrix with bi , i = 1, . . . , n, on its diagonal.  b 1 = ni=1 |bi |
Ma et. al investigated noncooperative behaviors of multi-agent is 1-norm of b. Let ei denote the canonical vector with a 1
systems with two competitive leaders in [34] and [35]. For a in the ith entry and 0s elsewhere. A matrix is positive (resp.
leader and its opponent, the authors considered the problem of nonnegative) if all its entries are positive (resp. nonnegative).
minimizing the influence of the opponent. The problem was for- For a square matrix A, adj A and det A are the adjugate and
mulate as three optimization problems [34]. In [35], Ma et al. the determinant of A, respectively. For two matrices A and B,
proposed a zero-sum game where two competitive leaders are A ⊗ B is Kronecker product of A and B. For two sets S1 and
players. Both of the two leaders want to maximize their influ- S2 , denote S1 × S2 as the Cartesian product and S1 \ S2 =
ence on the followers. The Nash equilibrium solutions are given S1 − S2 . Let In = {1, 2, . . . , n}.
when the followers’ interaction graph is a bidirected star graph
or a circulant graph.
Inspired by the aforementioned papers, we investigate com-
II. PRELIMINARIES
petition behavior of multi-agent systems, which consist of three
groups of agents: V1 , V2 , and V3 . Agents of V1 and V2 can A. Graph Theory
send information to at most m (≥ 1) agents of V3 . Then, they In this section, we present some basic notions of algebraic
can influence agents in V3 . Agents of V1 and V2 want to exert graph, which will be used in this paper. For more details, inter-
maximum influence on V3 , which leads to competition between ested readers are referred to [36] for a thorough study of graph
them. Since agents of V1 (V2 ) cooperate and reach consensus, theory.
they have the same interests. Consequently, the competition be- Let G = {V, E} be a directed graph consisting of a vertex set
tween agents of V1 and V2 is actually a competition between two V = {v1 , v2 , . . . , vn } and an edge set E = {(vj , vi ) ∈ V × V}.
groups—V1 and V2 . We can formulate a finite noncooperative In this paper, we assume that there are no self-loops. For an
game to analyze the competition between two groups. The main edge (vj , vi ), vj is called the parent vertex of vi . A directed tree
contribution of this paper is threefold. First, we develop a non- is a graph, where every vertex, except the root, has exactly one
cooperative game where V1 and V2 are two players and their in- parent. A directed path in a graph G is a sequence vi 1 , . . . , vi k
fluence power on V3 are payoffs. Because the players’ decisions of vertices such that for s = 1, . . . , k − 1, (vi s , vi s + 1 ) ∈ E. A
will determine the interaction topology of the system, seeking graph G is strongly connected if between every pair of distinct
Nash equilibrium solution of the game is equivalent to choos- vertices vi , vj , there is a directed path that begins at vi and ends at
ing equilibrium topology for the system. Second, the necessary vj . A = [aij ] is the adjacency matrix with aij = 1 if (vj , vi ) ∈
condition is established for equilibrium topology. It should be E and aij = 0 otherwise.
noticed that the game might have Nash equilibria, which are  The Laplacian matrix is defined by
L = [lij ] with lii = nj=1 aij and lij = −aij for i = j. It is
not interchangeable, whereas, when the graph of V3 is a tree
easy to see that L1n = 0.G is a bidirected graph, if AT = A.
or has a center vertex, exchangeable equilibrium topologies are
For a connected bidirected graph G, we have: 1) det L = 0; and
obtained. Furthermore, agents of V3 might reach consensus un-
2) adj L = τ (G)1n 1Tn , where τ (G) is the number of spanning
der the equilibrium topology, which is different from the existed
trees in graph G [36]. A spanning tree of G is a directed tree,
results of containment control. Third, the necessary and suffi-
which consists of all the vertices and some edges in G. A vertex
cient condition is obtained for the equilibrium topology when
is called the root vertex of G if it is the root of a spanning
the graph of V3 is bidirected. Moreover, for the scenario where
tree. Suppose that graph G has a spanning tree. Denote by V (r )
V3 is a bidirected circulant graph, we prove that all strategy
the set of root vertices. It is easy to see that, by arranging
pairs are interchangeable Nash equilibria. It is worth empha-
the indices of agent, the Laplacian matrix of G can be written
sizing that the current work differs from that in [35], which is
as L = ( LL( r f ) L(0f ) ), where L(r ) is the Laplacian matrix of the
(r )

extended to a general framework in this paper, mainly in the

following two points. First, in this paper, competition between graph induced by V (r ) . A vertex vi ∗ is called the central root
two groups is considered, whereas Ma et al. [35] focuses on the vertex of G if it satisfies: 1) V (r ) ={vi ∗ }; and 2) it is the parent
case of competition between two leaders. Second, in this paper, vertex of all other vertices.
the interaction graph of V3 is a direct graph, whereas in [35], Lemma 1: Suppose that G has a spanning tree. Δ(r ) and
(f )
the interaction graph of followers is bidirected. Δ are two nonnegative and nonzero diagonal matrices with
This paper is organized as follows. In Section II, we introduce same dimensions with L(r ) and L(f ) , respectively. Then, the
some notions of graph theory and noncooperative game, and following conditions are satisfied.
state our problem. In Sections III and IV, we present our main 1) L(r ) + Δ(r ) , L(f ) , and L(f ) + Δ(f ) are invertible.
results and give some illustrative examples. Some conclusions 2) (L(r ) + Δ(r ) )−1 is positive.
are drawn in Section V. 3) (L(f ) )−1 and (L(f ) + Δ(f ) )−1 are nonnegative.
4958 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 6, JUNE 2017

Proof: The proof is similar to that of [35, Lemma 2], and it 2) The strategy pair (s∗1 , s∗2 ) is a Nash equilibrium if and
is omitted.  only if s∗1 ∈ {s∗ ∈ S1 , mins 2 ∈S 2 u1 (s∗1 , s2 ) = U } and
s∗2 ∈ {s∗2 ∈ S2 , maxs 1 ∈S 1 u1 (s1 , s∗2 ) = U }.
3) All Nash equilibrium solutions are interchangeable.
B. Noncooperative Finite Games
C. Problem Statement
In this section, we recall basic theoretic notions of noncoop-
erative finite game following [37]. Consider a multi-agent system consisting of n1 + n2 + n3
A two-person noncooperative finite game is denoted by G = agents. The agents are categorized into three groups: V1 =
(1) (1) (1) (2) (2) (2)
(P, S, U ), where P = {P1 , P2 } is the set of players, S = S1 × {v1 , v2 , . . . , vn 1 }, V2 = {v1 , v2 , . . . , vn 2 }, and V3 =
S2 , Sk is the set of pure strategies of player Pk ∈ P , and U = (3) (3) (3)
{v1 , v2 , . . . , vn 3 }. The interaction of V1 is modeled by
(u1 , u2 ), uk : S → R is the payoff function of player Pk ∈ P . a directed graph G1 = (V1 , E1 ). Likewise, G2 = (V2 , E2 ) and
Each player makes decision to maximize its payoff function by G3 = (V3 , E3 ) are the interaction graphs of V2 and V3 , re-
considering the possible rational choice of the other player. A (1) (2)
spectively. Denote A1 = {aij }n 1 ×n 1 , A2 = {aij }n 2 ×n 2 , and
strategy pair (s1 , s2 ) ∈ S means that P1 and P2 independently (3)
choose strategies s1 ∈ S1 and s2 ∈ S2 , respectively. For player A3 = {aij }n 3 ×n 3 as the adjacent matrices of G1 , G2 , and G3 ,
P1 , ŝ1 (s2 ) is the best response strategy to a strategy s2 ∈ S2 , respectively. The following assumptions are made throughout
if u1 (ŝ1 (s2 ), s2 ) ≥ u1 (s1 , s2 ) holds for all s1 ∈ S1 . Likewise, this paper.
ŝ2 (s1 ) is the best response strategy to a strategy s1 ∈ S1 , if A1: G1 , G2 , and G3 have a spanning tree.
u2 (s1 , ŝ2 (s1 ) ≥ u2 (s1 , s2 ) holds for all s2 ∈ S2 . A2: There does not exist any information flow between V1
Definition 1: [37] A strategy pair (s∗1 , s∗2 ) is said to consti- and V2 , and from V3 to V1 or V2 .
tute a noncooperative (pure) Nash equilibrium solution for a A3: There exist information flows from V1 and V2 to V3 .
two-person nonzero-sum game G = (P, S, U ), if s∗1 is the best Remark 1: A1 describes the network structure of three
strategy to s∗2 , and vice versa. groups.A2 andA3 indicate the information interaction among
A pair of strategies (s1 , s2 ) ∈ S is said to be better than an- three groups.
other pair of strategies (s1 , s2 ) ∈ S, if u1 (s1 , s2 ) ≥ u1 (s1 , s2 ) According to A3, we define two matrices B and D to de-
and u2 (s1 , s2 ) ≥ u2 (s1 , s2 ) hold and at least one of these in- scribe information flows from V1 and V2 to V3 , respectively.
(1)
equalities is strict. A Nash equilibrium strategy pair is said to Define B = {bij }n 3 ×n 1 where bij = 1 if agent vj ∈ V1 sends
(3)
be admissible if there exists no better Nash equilibrium strategy information to agent vi ∈ V3 , while bij = 0 otherwise. Let
pair. Let (s1 , s2 ) and (s1 , s2 ) be two Nash equilibrium solutions. (2)
D = {dik }n 3 ×n 2 where dik = 1 if agent vk ∈ V2 sends infor-
(s1 , s2 ) and (s1 , s2 ) are said to be interchangeable, if (s1 , s2 ) (3)
mation to agent vi ∈ V3 , while dik = 0 otherwise.
and (s1 , s2 ) are also two Nash equilibrium solutions. When a
By A1 , we know that G3 has a root vertex set. Without loss
game has at least two admissible and noninterchangeable Nash
of generality, we assume that
equilibrium solutions, there exist dilemmas, i.e., each player’s (r ) (3) (3)
decision for seeking the maximum payoff may lead to a lower A4: V3 = {v1 , . . . , vn r } is the root vertex set of G3 .
(r ) (r )
payoff for both two players [37]. This kind of dilemmas cannot Denote G3 by the subgraph of G3 induced by V3 . It follows
be completely avoided unless changing the mechanism of the that the Laplacian matrix of G3 is
game, e.g., some cooperation is allowed, or there is a hierarchy  (r ) 
in decision making. L3 0
L3 = (1)
Let u1 , u2 be payoff functions of G1 , and w1 , w2 be payoff (r f ) (f )
L3 L3
functions of G2 . Two finite two-person games G1 and G2 are
said to be strategically equivalent, if 1) each player has the same (r ) (r )
strategy set, in both games; and 2) ui (s1 , s2 ) = αi wi (s1 , s2 ) + where L3 is the Laplacian matrix of G3 . As a result, we
βi , for all (s1 , s2 ) ∈ S1 × S2 , where αi > 0 and βi ∈ R, i ∈ have B = ( BB12 ) and D = ( D
D2 ), where B1 ∈ R
1 n r ×n 1
and D1 ∈
Rn r ×n 2 indicate the information flows from V1 and V2 to V3 ,
(r )
{1, 2} are constants.
A game G is said a two-person zero-sum game, if u1 (s1 , respectively.
s2 ) + u2 (s1 , s2 ) = 0 holds for all (s1 , s2 ) ∈ S. For two-person It is easy to see that all agents of V1 , V2 , and V3 consist of
zero-sum games, we refer to U = maxs 1 ∈S 1 mins 2 ∈S 2 u1 (s1 , a graph G = (V, E) where V = V1 ∪ V2 ∪ V3 , E = E1 ∪ E2 ∪
s2 ) and U = mins 2 ∈S 2 maxs 1 ∈S 1 u1 (s1 , s2 ). E3 ∪ {edges from V1 and V2 to V3 }. The Laplacian matrix of G
Lemma 2: [37] All strategically equivalent finite games have can be written as
the same set of Nash equilibria. Moreover, if the sum of payoff ⎛ ⎞
functions is a constant, then the game is strategically equivalent L1 0 0 0
⎜ 0 L2 ⎟
to a two-person zero-sum game. ⎜ 0 0 ⎟
⎜ ⎟ (2)
Lemma 3: [37] Suppose that a two-person zero-sum game ⎜ −B −D L(r ) + K ⎟
G satisfies U = U . Then, the following statements hold. ⎝ 1 1 3 0 ⎠
(r f ) (f )
1) G has a (pure) Nash equilibrium point. −B2 −D2 L3 L3 +H
MA et al.: NASH EQUILIBRIUM TOPOLOGY OF MULTI-AGENT SYSTEMS WITH COMPETITIVE GROUPS 4959

will converge to convex combinations of X(t), Y (t), and

Zr (t), i.e.,

n1
(1)

n2
(2)

nr
lim zk (t) = αk i xi (t) + αk j yj (t) + βk s zs (t)
t→∞
i=1 j =1 s=1

k = nr + 1, nr + 2, . . . , n3
where
⎧  (1) (1)

⎪
⎪ αk 1 , . . . , αk ,n 1
Fig. 1. Graph G decided by B and D. ⎪
⎪
⎪
⎪  −1
⎪
⎪ (f )
⎪
⎪ = e T
k −n r L + diag{B 1 + D 1 } B2
⎪
⎪ 
3

2 n1 2 n2
⎪
⎪
where Li is the Laplacian matrix of Gi , i = 1, 2, K = diag{B1 ⎪
⎨ αk(2) (2)
1 , . . . , αk ,n 2
1n 1 + D1 1n 2 }, H = diag{B2 1n 1 + D2 1n 2 }.  −1
⎪
⎪
Remark 2: If G1 , G2 , and G3 are given, we can easily find ⎪
⎪ = e T
L
(f )
+ diag{B 1 + D 1 } D2
⎪
⎪
k −n r 3 2 n 1 2 n 2
that G is decided by B and D (see Fig. 1). Moreover, we have ⎪
⎪
⎪
⎪ [βk 1 , . . . , βk ,n r ]
following conditions. ⎪
⎪
⎪
⎪  −1
(r )
1) If V1 and V2 do not send information to V3 , then B1 = ⎪
⎩ = eT (f ) (r f )
0n r ×n 1 and D1 = 0n r ×n 2 . k −n r L3 + diag{B2 1n 1 + D2 1n 2 } L3 .
2) If there exists at least one agent of V1 (V2 ) who sends (5)
(r )
information to V3 , then B1 = 0n r ×n 1 (D1 = 0n r ×n 2 ). 2) Assume that B1 = 0n r ×n 1 or D1 = 0n r ×n 2 . Then, agents
Let xi (t) ∈ R, yj (t) ∈ R, and zk (t) ∈ R be the states of of V3 will converge to convex combinations of X(t) and
(1) (2) (3) Y (t), i.e.,
vi ∈ V1 , vj ∈ V2 , and vk ∈ V3 , respectively. The dynamics
of agents are represented by 
n1
(1)

n2
(2)
⎧ lim zk (t) = αk i xi (t) + αk j yj (t),
(x) t→∞
⎪
⎪ ẋ = ui , i ∈ In 1 j =1 j =1
⎨ i
(y )
ẏj = uj , j ∈ In 2 (3) i = 1, 2, . . . , n3
⎪
⎪
⎩ (z )
żk = uk , k ∈ In 3 where
⎧ 
(x) (y ) (z ) (1) ⎪
⎪
(1) (1)
αk 1 , . . . , αk ,n 1 = eTk (L3 + diag{B1n 1
where ui , uj , uk ∈ R are control protocols of agents vi , ⎪
⎪
(2) (3)
⎪
⎨ + D1n 2 })−1 B
vj , and vk , respectively. In this paper, we propose the fol-
  (6)
lowing control protocols: ⎪
⎪ (2) (2)
⎧ ⎪
⎪ αk 1 , . . . , αk ,n 2 = eTk (L3 + diag{B1n 1
 1 (1) ⎪
⎩
⎪
⎪
(x)
ui = nh=1 aj h (xh − xi ), i ∈ In 1 + D1n 2 })−1 D.
⎪
⎪
⎪
⎪ 
⎪
⎨ u(y ) = n 2 a(2) (yh − yj ), Remark 3: From (5) and (6), we find that the convergence
j h=1 j h j ∈ In 2
states of V3 can be written as
⎪
⎪ (z )  (3) 
⎪
⎪ u = nh=1 3
ak h (zh − zk) + nh=1 1
bk h (xh − zk) 
n1 
n2 
nr
⎪ k
⎪
⎪
⎩  lim zk (t) =
(1)
αk i xi (t) +
(2)
αk j yj (t) + βk s zs (t),
+ nh=1 2
dk h (yh − zk ), k ∈ In 3 . t→∞
i=1 j =1 s=1
(4)
Define vector notations X = [x1 , . . . , xn 1 ]T , Y = [y1 , . . . , i = 1, 2, . . . , n3 . (7)
yn 2 ]T , and Z = [ZrT , ZfT ]T , where Zr = [z1 , . . . , zn r ]T and
When B1 = 0n r ×n 1 and D1 = 0n r ×n 2 , we have
Zf = [zn r +1 , . . . , zn 3 ]T . By [10, Th. 1], the following results ⎧ n r
are given. ⎨ αk(1) (2)
i = 0, αk j = 0, s=1 βk s = 1, k ∈ In r , i ∈ In 1 , j ∈ In 2
Lemma 4: Suppose that A1–A4 hold. Then, agents of V1 and ⎩ n 1 α(1) + n 2 α(2) + n r β = 1, k ∈ I \I
agents of V2 achieve consensus, respectively, i.e. i=1 k i j =1 k j s=1 k s n3 nr
(8)
lim X(t) = 1n 1 f1T X(0) and lim Y (t) = 1n 2 f2T Y (0) whereas if B1 = 0n r ×n 1 or D1 = 0n r ×n 2 , then
t→∞ t→∞

where f1 ∈ R and f2 ∈ R are the left eigenvector of L1

n1 n2 
n1
(1)

n2
(2)
and L2 corresponding to eigenvalue 0, respectively. αk i + αk j = 1, βk s = 0, k ∈ In 3 , s ∈ In r . (9)
i=1 j =1
1) Assume that B1 = 0n r ×n 1 and D1 = 0n r ×n 2 . Agents
(r )
of V3 achieve consensus, i.e., limt→∞ Zr (t) = 1n r frT By (7), we have the convergence state of agent vk decided
(3)
(r )
Zr (0), where fr ∈ Rn r is the left eigenvector of L3 by xi (t), yj (t), and zk (t), i ∈ In 1 , j ∈ In 2 , k ∈ In r . Thus, it
(3) (3) (1) (2)
corresponding to eigenvalue 0. Agents vn r +1 , . . . , vn 3 is easy to know that coefficients αk i and αk j measure how
4960 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 6, JUNE 2017

agents of vi
(1) (2)
and vj
(3)
exert influence on the agent vk , re- Case 1: If B1 = 0n r ×n 1 and D1 = 0n r ×n 2 , by Lemma 4 and
 3 (1)  3 (2) (8), we have
spectively. Therefore, nk =1 αk i ( nk =1 αk j ) represents agent
⎛ ⎞
(1) (2)
vi ’s (vj ’s) influence on V3 . According to Lemma 4, agents n3 n1 n2 
nr

of V1 (V2 ) will reach consensus, which implies that they have ⎝ (1)
αk i +
(2)
αk j + βk s ⎠ = n3 .
 1 n 3 (1)
the same interest of maximizing the influence. ni=1 k =1 αk i
k =1 i=1 j =1 s=1
n 2 n 3 (2)
( j =1 k =1 αk j ) can be employed to measure the influence Consequently,
 1 n 3 (1)
power of V1 (V2 ) on V3 . Note that the greater ni=1 k =1 αk i
n 2 n 3 (2)

n3 
nr
( j =1 k =1 αk j ) is, the more powerful influence of V1 (V2 ) u1 (B, D) + u2 (B, D) = n3 − βk s ≤ n3 − nr < n3 .
is. Since an agent of V1 and an agent of V2 have different conver- k =1 s=1
gence states, they influence agents of V3 by different manners. (11)
This produces conflict between V1 and V2 . Inspired by this fact, Case 2: If B1 = 0n r ×n 1 or D1 = 0n r ×n 2 , by Lemma 4 and (9),
we propose the following noncooperative finite game where V1 we have
and V2 are two competitive player.
u1 (B, D) + u2 (B, D) = n3 . (12)
Definition 2: For multi-agent system (3) and (4), we define
the following. Thus, it follows from (11) and (12) that (10) holds. 
1) Players: Let V1 and V2 be two players, i.e., P = Property 2: Suppose that G3 is strongly connected. Then, the
{V1 , V2 }. Players make their decisions independently and following statements hold.
simultaneously. Meanwhile, each one unilaterally seeks 1) Game G is equivalent to a zero-sum game.
the maximum payoff, by also taking into account the 2) All Nash equilibrium solutions are interchangeable.
possible rational choice of the other player. Proof: Since the interaction graph of V3 is strongly con-
2) Strategies: Each player choose at most m pairs of (r )
nected, we have V3 = V3 . Hence, we have B1 = 0n r ×n 1 and
(r ) (3) (r )
agents (vj , vk )(r = 1, 2) where vj will send in- D1 = 0n r ×n 2 . As a result,
(3)
formation to vk . In other words, the strategy sets
of V1 and V2 are S1 = {B | 1Tn 3 B1n 1 ≤ m} and S2 = u1 (B, D) + u2 (B, D) = n3 (13)
{D | 1Tn 3 D1n 2 ≤ m}, respectively. holds for each strategy pair (B, D) ∈ S1 × S2 , which means
3) Payoffs: Payoff functions of V1 and V2 are u1 (B, D) =
n 1 n 3 (1) n 2 n 3 (2)
that the sum of payoff functions is a constant. Therefore, it
i=1 k =1 αk i and u2 (B, D) = j =1 k =1 αk j , re- follows from Lemma 2 that game G is equivalent to a zero-sum
spectively. game. By Lemma 3, we can obtain straightforward that all Nash
We denote this game as G = (P, S, U ), where S = S1 × S2 equilibrium solutions are interchangeable. 
and U = (u1 , u2 ). Remark 6: By Property 2, u1 (B, D) + u2 (B, D) < n3
Remark 4: In game G, players are two groups V1 and V2 . when B1 = 0n r ×n 1 and D1 = 0n r ×n 2 . It means that the sum of
A strategy pair (B, D) corresponds with an interaction graph of payoffs varies with strategy pair. Consequently, the game may
multi-agent system (3)–(4) (see Fig. 1). Let this graph be G(B,D) . not be a zero-sum game. By Property 3, we obtain that the game
The Laplacian matrix of G(B,D) is written in (2). is equivalent to a zero-sum game when G3 is strongly connected.
Remark 5: This game is common in real world. Consider two Recalling [35], the graph of followers is undirected connected
political parties run for election. The decision of a voter might and the game is equivalent to a zero-sum game. Hence, we
be influenced by his/her neighbors in social networks. Aiming extend the work of [35] to a general framework in this paper.
to win the election, each party reach consensus among their Property 3: Suppose that strategies B, B ∈ S1 and D, D ∈
members. Meanwhile, they choose some members to broadcast S2 .
their political opinion to some voters who will exercise a great 1) If B1n 1 = B 1n 1 and D1n 2 = D 1n 2 , then
influence on others.
u1 (B, D) = u1 (B , D) and u2 (B, D) = u2 (B, D ).
(14)
III. PROPERTIES OF GAME G 2) If B1n 1 = D 1n 2 and D1n 2 = B 1n 1 , then
In the following context, we will develop some properties of
u1 (B, D) = u2 (B , D ). (15)
game G.
Property 1: The sum of two players’ payoffs is not greater 3) If B1n 1 = D1n 2 , then
than n3 .
Proof: It suffices to prove u1 (B, D) = u2 (B, D). (16)

Moreover, if B1 1n 1 = D1 1n 2 = 0n r , then the agents of

u1 (B, D) + u2 (B, D) ≤ n3 (10)
group V3 will reach consensus asymptotically to

holds for all (B, D) ∈ S1 × S2 . Recalling Lemma 4, there are 1 

lim (x1 (t) + y1 (t)) . (17)
two cases that should be considered. 2 t→∞
MA et al.: NASH EQUILIBRIUM TOPOLOGY OF MULTI-AGENT SYSTEMS WITH COMPETITIVE GROUPS 4961

Proof: It follows from (5) and (6) that lemma in [38], we obtain

u1 (B, D) = Q−1 ej eTj Q−1

(Q + diag{ej })−1 − Q−1 = . (20)
1 + eTj Q−1 ej
⎧  −1
⎪
⎪
(f )
1Tn 3 −n r L3 + diag{B2 1n 1 + D2 1n 2 } B2 1n 1 ,
⎪
⎪ It follows from Lemma 1 that Q−1 and (Q + diag{ej })−1
⎪
⎪
⎨ for B = 0 are nonnegative matrices. Together with (20), we can obtain
1 n r ×n 1 and D1 = 0n r ×n 2
(18) that 1Tn 3 [(Q + diag{ej })−1 − Q−1 ]B1n 1 ≥ 0 and 1Tn 3 (Q +
⎪
⎪ −1
⎪
⎪ 1 T
(L 3 + diag{B1 n + D1n }) B1n , diag{ej })−1 ej > 0. Hence, u1 (B , D) > u1 (B, D).
⎪
⎪
n3 1 2 1
⎩ Case 2: We assume that B1 = B1 = 0n r ×n 1 and D1 =
for B1 = 0n r ×n 1 or D1 = 0n r ×n 2
0n r ×n 2 . Similar to the proof of case 1, we can prove that
and u1 (B , D) > u1 (B, D).
Case 3: Suppose that B1 = 0n r ×n 1 , D1 = 0n r ×n 2 , and B1
u2 (B, D) = = 0n r ×n 1 . It follows that B = ( BB12 ) and B1 1n 1 = pj , where pj
⎧  −1 is an nr -dimensional canonical vector with 1 in the jth entry and
⎪
⎪ 1T
L
(f )
+ diag{B 1 + D 1 } D2 1n 2 , 0s elsewhere. Therefore, we get L3 + diag{B 1n 1 + D1n 2 } =
⎪
⎪ n 3 −n r 3 2 n1 2 n2
⎪
⎪ ( L(Rr f ) 0 n r ×(Qn 3 −n r ) ) and (L3 + diag{B 1n 1 + D1n 2 })−1 =
⎨ for B = 0
n r ×n 1 and D1 = 0n r ×n 2
3
1 −1 0 n r ×( n 3 −n r ) (r )
(19) ( −Q −1 RL( r f ) R −1 ), where R = L3 + diag{pj } and
⎪
⎪ −1 Q −1
⎪ 1 D1 }) D1
T 3
⎪ n3 (L + diag{B1 + , (f )
⎪
⎪
3 n 1 n 2 n 1
Q = L3 + diag{B2 1n 1 + D2 1n 2 }. Together with (18), we
⎩
for B1 = 0n r ×n 1 or D1 = 0n r ×n 2 . have
u1 (B , D)− u1 (B, D) = 1Tn r R−1 pj − 1Tn 3 −n r Q−1 L3
(r f ) −1
Thus, we can straightforward obtain that (14)–(16) hold. R pj .
 1 (1)
By Lemma 4, we have ni=1 αk i = eTk (L3 + diag{B1n 1 + According to Lemma 1, we know that R−1 is a positive ma-

D1n 2 })−1 B1n 1 , j =1 αk j = eTk (L3 + diag{B1n 1 +D1n 2})−1
n (2)
trix and Q−1 is a nonnegative matrix. As a result, we have
2

n 1 (1) n 2 (2)
1Tn r R−1 pj > 0 and −1Tn 3 −n r Q−1 L3 R−1 pj ≥ 0, which im-
(r f )
D1n 2 , i=1 αk i + j =1 αk j = 1, and
plies that u1 (B , D) > u1 (B, D).
lim x1 (t) = lim x2 (t) = · · · = lim xn 1 (t) Similar to the above proof, we can prove that u2 (B, D ) >
t→∞ t→∞ t→∞
u2 (B, D) holds for all B ∈ S1 . 
lim y1 (t) = lim y2 (t) = · · · = lim yn 2 (t). By Property 4, we can straightforward obtain the following
t→∞ t→∞ t→∞
result.
n 1 (1) n 2 (2)
Therefore, we know that i=1 αk i =
1
j =1 αk j = 2 if Property 5: A best response strategy of game G always con-
B1n 1 = D1n 2 = 0, which implies that (17) holds.  tains m agents of V3 .
Property 4:
1) Suppose that player V2 do not change its strategy. Then, IV. EQUILIBRIUM TOPOLOGIES
player V1 can increase its payoff by adding new edges
According to Remark 4, a strategy pair (B, D) corresponds to
from V1 to V3 .
an interaction graph G(B,D) . Therefore, the following definition
2) Suppose that player V1 do not change its strategy. Then,
is given.
player V2 can increase its payoff by adding new edges
Definition 3: If a strategy pair (B ∗ , D∗ ) is a Nash equilibrium
from V2 to V3 .
solution of game G, then the corresponding interaction graph
Proof: For a strategy B ∈ S1 , let B be the new strategy,
G(B∗ ,D∗ ) is called the equilibrium topology of system (3)–(4).
which is obtained by adding new edges from V1 to V3 . Likewise,
for a strategy D ∈ S2 , let D be the new strategy gotten by
A. Necessary Condition for Being an Equilibrium
adding new edges from V2 to V3 . It follows that B − B and
D − D are nonzero nonnegative matrices. It suffices to prove Topology
u1 (B , D) > u1 (B, D) and u2 (B, D ) > u2 (B, D) hold for all Theorem 1: Suppose that G(B∗ ,D∗ ) is the equilibrium topol-
B ∈ S1 , D ∈ S2 . ogy. Then, the following statements hold.
Since B − B is a nonzero nonnegative matrix, we have 1) G(B∗ ,D∗ ) contains m edges that begin at V1 and end at V3 ,
(B − B)1n 1 = ei 1 + · · · + ei k . Without loss of generality, we and m edges that begin at V2 and end at V3 .
assume that (B − B)1n 1 = ej . Recalling Lemma 4, we will 2) G(B∗ ,D∗ ) contains at least one edge that begins at V1 or V2
consider three situations: 1) B1 = 0n r ×n 1 or D1 = 0n r ×n 2 ; 2) and ends at the root vertex set of G3 .
B1 = B1 = 0n r ×n 1 and D1 = 0n r ×n 2 ; and 3) B1 = 0n r ×n 1 , Proof: First, we know that B ∗ is the best response of D∗ , and
D1 = 0n r ×n 2 and B1 = 0n r ×n 1 . vice versa. By Property 5, it follows that G(B∗ ,D∗ ) contains m
Case 1: We assume that B1 = 0n r ×n 1 or D1 = 0n r ×n 2 . edges that begin at V1 and end at V3 , and m edges that begin at
It follows from (18) that u1 (B , D) − u1 (B, D) = 1Tn 3 [(Q + V2 and end at V3 .
diag{ej })−1 − Q−1 ]B1n 1 + 1Tn 3 (Q + diag{ej })−1 ej , where Second, we assume that neither B ∗ nor D∗ contains root
Q = L3 + diag{B1n 1 + D1n 2 }. By the matrix inversion vertices of G3 . According to the definition of Nash equilibrium
4962 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 6, JUNE 2017

and U2 = [u2 (ei , ej )]8×8 = U1T . It is easy to see that this game
has two Nash equilibria (e1 , e2 ) and (e2 , e1 ). It is shown that
two Nash equilibria contain vertex 1, which is the root vertex of
G3 . This result illustrates the effectiveness of theoretical results
in Theorem 1. Since u1 (e1 , e2 ) < u1 (e2 , e1 ) and u2 (e1 , e2 ) >
u2 (e2 , e1 ), we have G admits two admissible Nash equilibrium
solutions, which are obviously not interchangeable.
Fig. 2. Directed graph G3 for Example 1. Remark 7: In Example 1, it is shown that G has multiple ad-
missible Nash equilibrium solutions, which are not interchange-
able. Considering that there is not cooperation between players
solution, we have by the nature of the problem, two players make decisions inde-
 pendently and simultaneously. Hence, V1 might stick to e2 , and
u1 (B ∗ , D∗ ) ≥ u1 (B, D∗ ), B ∈ S1
(21) V2 might also adopt e2 , thus yielding an outcome of (e2 , e2 ),
u2 (B ∗ , D∗ ) ≥ u2 (B ∗ , D), D ∈ S2 . which is not a Nash equilibrium solution. This is indeed one
of dilemmas of noncooperative nonzero-sum decision making
Assume that (B , D ) ∈ S satisfies B 1n 1 = D∗ 1n 2 and D 1n 2
[37]. In this example, the reason of being a dilemma is that Nash
= B ∗ 1n 2 . By Property 3, it is easy to know that (B , D ) is also
equilibrium solutions of game G are not interchangeable. There
a Nash equilibrium solution. Therefore, we have
 is really no remedy for it unless changing the mechanism of the
u1 (B , D ) ≥ u1 (B, D ), B ∈ S1 game, for instance, players are admitted to communicate and
(22) negotiate before making decisions when facing such dilemmas.
u2 (B , D ) ≥ u2 (B , D), D ∈ S2 .
Although Example 1 exists multiple admissible Nash equi-
Assume that strategy B̂ contains a root agent of G3 . It follows librium solutions that are not interchangeable, Nash equilibrium
that B̂1 = 0n r ×n 1 . By Property 1, we have solutions might be interchangeable under some situations. In the
next content, we will give some special cases where the game
u1 (B ∗ , D∗ ) + u2 (B ∗ , D∗ ) = n3 − nr < n3 = u1 (B̂, D∗ ) has interchangeable Nash equilibria.
+ u2 (B̂, D∗ ).
B. Equilibrium Topologies Under Some Special Cases
Consequently, we obtain that
1) G3 is a Directed Tree:
∗ ∗ ∗
u1 (B , D ) < u1 (B̂, D ) (23) Theorem 2: Suppose that G3 is a directed tree and m = 1.
Then, the following statements hold.
or
1) Graph G(B,D) is the equilibrium topology, if and only if
u2 (B ∗ , D∗ ) < u2 (B̂, D∗ ). (24) both of two strategies contain the root vertex of G3 .
2) All Nash equilibrium solutions are interchangeable.
When inequality (23) holds, it conflicts with (21). Let D̂ (3)
Proof: Without loss of generality, we assume that v1 is
∈ S2 satisfy D̂1n 2 = B̂1n 1 . When (24) holds, it follows that ∗ ∗ (3)
the root vertex of G3 . Suppose that B and D contain v1 .
u1 (B , D ) < u1 (B , D̂). Since u1 (B , D ) + u2 (B , D ) = n3 ∗ ∗
It follows that B 1n 1 = D 1n 2 = [1 0 0 . . . 0] . By Property 3,
T
− nr and u1 (B , D̂) + u2 (B , D̂) = n3 , we have u2 (B , D ) <
we have u1 (B ∗ , D∗ ) = u2 (B ∗ , D∗ ) = n23 . Thus, from the
u2 (B , D ) + nr < u2 (B , D̂), which conflicts with (22). Con-
definition of Nash equilibrium solution, it is suffices to prove
sequently, G(B∗ ,D∗ ) contains at least one edge that begins at V1
u1 (B, D∗ ) ≤ n23 and u2 (B ∗ , D) ≤ n23 hold for all B ∈ S1 ,
or V2 and ends at the root vertex set of G3 .  (3)
Theorem 1 proposes a necessary condition for a strategy pair D ∈ S2 . Suppose that strategy B contains vi . Then, we have
(B ∗ , D∗ ) being Nash equilibrium solution.
(3)
B1n 1 = ei . Let Ti = {vj ∈ V3 | there exists a path begins at
(1) (2)
Example 1: Suppose V1 = {v1 }, V2 = {v1 }, and V3 = (3) (3)
vi and ends in vj } and |Ti | = ki , where |·| is the cardinality
{1, 2, 3, 4, 5, 6, 7, 8}. G3 is shown in Fig. 2. Therefore, strategy of a set. Since G3 is a tree, we have { kkii =n 3 −1,i=1,
< n 3 −1,i= 1 . If i =
sets of V1 and V2 are S1 = S2 = {ei , i = 1, 2, . . . , 8}, where ei
is the canonical vector of R8 . By using (18) and (19), we have 1, it is obviously to see that u1 (B, D∗ ) = k i 2+1 = n23 . For the
case of i = 1, by Theorem 1 in [10], we know limt→∞ zj (t) =
U1 = [u1 (ei , ej )]8×8 1 1
1 (t), j ∈T i ∪{i}
{ 2 x 1 (t)+x21 y(t), for strategy pair (B, D∗ ). It follows that
⎛ ⎞ else
4 3.67 4.5 4.5 4.5 4.5 4.5 4.5 u1 (B, D∗ ) = k i 2+1 < n23 . Therefore, we have u1 (B, D∗ ) ≤ n23
⎜4.33 2.6 3 3 3 3 3 3.8 ⎟ holds for all B ∈ S1 . Similarity, it is easy to prove that
⎜ ⎟
⎜ 3.5 2 2 2.33 2.33 2.33 2.33 3 ⎟ u2 (B ∗ , D) ≤ n23 holds for all D ∈ S2 . Thus, together with The-
⎜ ⎟
⎜ 3.5 2 2.33 2 2.33 2.33 2.33 3 ⎟ orem 1, we have: 1) (B, D) is a Nash equilibrium solution if
=⎜⎜ 3.5
⎟
⎜ 2 2.33 2.33 2 2.33 2.33 3 ⎟⎟
(3)
both of two strategies contain v1 ; and 2) (B, D) is not a Nash
⎜ 3.5 2 2.33 2.33 2.33 2 2.33 3 ⎟
⎜ ⎟ equilibrium solution if at least one strategy does not contain
⎝ 3.5 2 2.33 2.33 2.33 2.33 2 3 ⎠ (3)
v1 . Therefore, a strategy pair is a Nash equilibrium if and only
3.5 1.6 2 2 2 2 2 2.33 if both of the two strategies contain the root agent of G3 .
MA et al.: NASH EQUILIBRIUM TOPOLOGY OF MULTI-AGENT SYSTEMS WITH COMPETITIVE GROUPS 4963

It is obvious that all Nash equilibrium solutions are interchange-

able. Consider a Nash equilibrium solution
⎛⎛ ⎞ ⎛ ⎞⎞
10 10
⎜⎜ 0 0 ⎟ ⎜ 0 0 ⎟⎟
(B ∗ , D∗ ) = ⎜ ⎜ ⎟ ⎜ ⎟⎟
⎝⎝ 0 0 ⎠, ⎝ 0 0 ⎠⎠ .
00 00
The corresponding interaction graph is shown in Fig. 3(a). The
state trajectories of agents are presented in Fig. 3(b), where
agents of V3 achieve consensus. Those results illustrate the ef-
fectiveness of theoretical results in Theorem 2 and Corollary 1.
2) G3 has a Central Root Vertex:
Theorem 3: Suppose that G3 has a central root vertex and
m = 1. Then, the following statements hold.
1) Graph G(B,D) is the equilibrium topology if and only if
both of two strategies contain the central root vertex.
2) All Nash equilibrium solutions are interchangeable.
Proof: Without loss of generality, we assume that the cen-
tral root vertex is v1 . We need to prove that (B ∗ , D∗ ) is a
Fig. 3. (a) Interaction graph G(B∗ , D∗ ) and (b) state trajectories of all the (3)
agents for Example 2. Nash equilibrium if and only if B ∗ 1n 1 = D∗ 1n 2 = e1 . Accord-
ing to the definition of Nash equilibrium, it suffices to prove
Let (B1∗ , D1∗ ) and (B2∗ , D2∗ ) be two Nash equilibrium solutions. u1 (B ∗ , D∗ ) ≥ u1 (B, D∗ ) and u2 (B ∗ , D∗ ) ≥ u2 (B ∗ , D) hold for
We have B1∗ , B2∗ , D1∗ , and D2∗ that contain the root agent of G3 . all B ∈ S1 and D ∈ S2 .
(3)
Consequently, we observe that (B1∗ , D2∗ ) and (B1∗ , D2∗ ) are also Let strategies B and D contain vj (j = 1). This implies
two Nash equilibrium solutions, which means that (B1∗ , D1∗ ) and (3)
B1n 1 = D1n 2 = ej . Since v1 is a central root vertex, we have
(B2∗ , D2∗ ) are interchangeable.  0 Tn
L3 = ( −1 n0 3 −1
). As a result, u1 (B ∗ , D∗ ) = 12 [1 + 1Tn 3 −1
Corollary 1: Suppose that G3 is a tree and m = 1. Then, 3 −1 L3
(f )

agents of V3 will achieve consensus under the equilibrium topol- (L3 )−1 1n 3 −1 ] and u1 (B ∗ , D) = 1 + 1Tn 3 −1 (L3 + diag
(f ) (f )

ogy. {pj −1 })−1 1n 3 −1 , where pj −1 is a (n3 − 1)-dimensional

Proof: By Property 3 and Theorem 2, we can obtain this canonical vector with 1 in the j − 1th entry and 0s elsewhere.
result straightforward.  It follows that
Remark 8: A tree structural graph means that there exist
a hierarchy among agents. In a hierarchical system, the most u1 (B ∗ , D) − u1 (B ∗ , D∗ )
  −1
powerful agent is the root agent. As a result, each player will 1 (f )
choose this agent to propagate its information. Therefore, the = 1 + 1Tn 3 −1 L3 + diag{pj −1 } 1n 3 −1
2
results of Theorem 2 seems intuitional.  −1  −1  
Example 2: G1 , G2 , and G3 are shown in Fig. 3(a). It is obvi- (f )
+ 1Tn 3 −1 L3 + diag{pj −1 } − L3
(f )
1n 3 −1 .
ous that G3 is a directed tree. Assume that V1 and V2 can only
select one agent in V3 to connect, respectively, i.e., m = 1. From the matrix inversion lemma [38], we obtain
Therefore, strategy sets of V1 and V2 are S1 = S2 = {B =  −1  −1
[bij ]4×2 , bij ∈ {0, 1}, 1T4 B12 = 1}. We know that each player (f )
L3 + diag{pj −1 } − L3
(f )

has eight strategies. Therefore, we have

 −1  −1
⎛ ⎞ L3
(f )
pj −1 pTj−1 L3
(f )
2 1.5 0.5 0.5  
⎜2.5
=  −1 .
1 0.5 0.5 ⎟
⎟⊗ 1 1
=⎜
(f )
U1 = [u1 (B, D)]8×8 1 + pTj−1 L3 pj −1
⎝3.5 1.25 0.33 0.5 ⎠ 11
3.5 1.25 0.5 0.33 By Lemma 1, we know that (L3 )−1 and (L3 + diag
(f ) (f )

{pj −1 })−1 are nonnegative matrices. Consequently, we

and U2 = [u2 (B, D)]8×8 = U1T . It is easy to see that this game have u1 (B ∗ , D) > u1 (B ∗ , D∗ ). Since B ∗ 1n 1 = D∗ 1n 2 = e1
has four Nash equilibrium solutions. A strategy pair (B, D) is a and B1n 1 = D1n 2 = ej , it follows from Properties 1
Nash equilibrium solution, if and only if B ∈ S ∗ and D ∈ S ∗ , and 3 that u1 (B ∗ , D∗ ) = n23 and u1 (B, D∗ ) = u2 (B ∗ , D) =
where n3 − u1 (B ∗ , D) < u1 (B ∗ , D∗ ). Likewise, we can prove that
⎧⎛ ⎞ ⎛ ⎞⎫ u2 (B ∗ , D) < u2 (B ∗ , D∗ ). The above proof yields the follow-
⎪
⎪ 10 01 ⎪ ⎪
⎨⎜ ⎜ 0 0 ⎟⎬ ing observation: 1) (B, D) is a Nash equilibrium solution if both
∗ ⎜ 0 0⎟⎟ ⎜ ⎟
S = ⎝ , . (3)
of two strategies contain v1 ; and 2) (B, D) is not a Nash equi-
⎪
⎪ 0 0 ⎠ ⎝ 0 0 ⎠⎪ ⎪
⎩ ⎭ (3)
librium solution if at least one strategy does not contain v1 .
00 00
4964 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 6, JUNE 2017

We find that all strategy pairs satisfyingB13 = D15 = e1 are

the interchangeable Nash equilibrium solutions. This is con-
sistent with theoretical results in Theorem 3. Let us consider
the Nash equivalent solution (B ∗ , D∗ ). The corresponding in-
teraction graph G(B∗ ,D∗ ) is described in Fig. 4(a). The states of
agents are presented in Fig. 4(b). We see that consensus can be
achieved among agents of V3 . This illustrates the effectiveness
of the conclusion of Corollary 2.
3) G3 is a Bidirected Graph:
In what follows, we assume that
A5. G3 is a connected bidirected graph and m = 1.
Let E = {ek , k ∈ In 3 | (L3 + diag{el })−1 ek 1 ≥ (L3 +
diag{ek })−1 el 1 , l ∈ In 3 }.
Theorem 4: Suppose that A5 holds. Then, graph G(B,D) is the
equilibrium topology, if and only if B1n 1 ∈ E and D1n 2 ∈ E.
Proof: For a strategy pair (B, D), denote B1n 1 = ei and
D1n 2 = ej . From (18), we obtain u1 (B, D) = 1Tn 3 [L3 +
Fig. 4. (a) Interaction graph G(B∗ , D∗ ) and (b) state trajectories of all the diag{ei + ej }]−1 ei . Let det[L3 + diag{ej }] = τ . It follows
agents for Example 3. from adj[L3 + diag{ei + ej }]ei = adj[L3 + diag{ej }]ei that
τ 1 Tn [L3 +diag{e j }] −1 e i
u1 (B, D) = det[L 3 +diag{e i +e j }]
. Since [L3 + diag{ej }]−1 is
a nonnegative matrix, we have
Therefore, (B ∗ , D∗ ) is a Nash equilibrium if and only if both of
 
two strategies contain the central root vertex.  
τ [L3 + diag{ej }]−1 ei 
Similar to the proof of Theorem 2, we can prove that all Nash u1 (B, D) = 1
. (25)
equilibrium solutions are interchangeable.  det [L3 + diag{ei + ej }]
Remark 9: The results in Theorem 3 is reasonable since the
Likewise, we have
center root agent can send information to all other agents, which
 
leads to its biggest influence power in G3 . Thus, both of the two  
τ [L3 + diag{ei }]−1 ej 
players choose this agent to spread their information. u2 (B, D) = 1
. (26)
Remark 10: It should be mentioned that there is an intersec- det [L3 + diag{ei + ej }]
tion of case 1 and case 2. For example, a directed star graph is
Recalling (13), it is easy to get that B1n 1 ∈ E if and only
a directed tree with center root vertex.
if B ∈ S1∗  {B ∈ S1 | minD∈S2 u1 (B, D) = n23 },D1n 1 ∈ E if
Corollary 2: Suppose that G3 has a central root vertex and
m = 1. Then, agents of V3 will achieve consensus under the and only if D ∈ S2∗  {D ∈ S2 | minB∈S1 u2 (B, D) = n23 }.
equilibrium topology. By Lemma 2, game G is equivalent to a zero-sum game G =
Proof: By Property 3 and Theorem 3, we can obtain this (P, S, W ) where W = (w1 , w2 ), w1 (B, D) = n13 u1 (B, D),
result straightforward.  and w2 (B, D) = − n13 w1 (B, D) = n13 u2 (B, D) − 1.Therefore,
Remark 11: According to previous theoretical results of it suffices to prove that (B, D) is a Nash equilibrium for G if
multi-agent systems, agents of V3 cannot reach consensus. How- and only if B ∈ S1∗ and D ∈ S2∗ . From the definition of S1∗ , we
ever, by Corollaries 1 and 2, we know that agents of V3 might have w1 (B, D) ≥ 12 for all B ∈ S1∗ , D ∈ S2 , which implies that
reach consensus under equilibrium topologies. The main reason U ≥ 12 . Similarly, w1 (B, D) ≤ 12 for all D ∈ S2∗ and B ∈ S1 ,
might be the competition between V1 and V2 . thereby resulting in U ≤ 12 . On the other hand, from the defini-
Example 3: The interaction graphs G1 , G2 , and G3 are de- tions of U and U , we know that U ≥ U . Consequently, it fol-
picted in Fig. 4(a). It is not difficult to observe that G3 lows that U = U = 12 , S1∗ = {B ∈ S1 , minD∈S2 w1 (B ∗ , D) =
has a central root vertex. Suppose that V1 and V2 can se- U } and S2∗ = {D ∈ S2 , maxB∈S1 w1 (B, D) = U }. By Lemma
lect one agent in V3 to connect, respectively, i.e., m = 1. 3, we know that (B, D) is a Nash equilibrium for G if and only
It follows that S1 = {B = [bij ]5×3 , bij ∈ {0, 1}, 1T5 B13 = 1} if B ∈ S1∗ and D ∈ S2∗ . Equivalently, graph G(B,D) is a Nash
and S2 = {D = [dij ]5×5 , dij ∈ {0, 1}, 1T5 D15 = 1}. By com- equilibrium topology if and only if B1n 1 ∈ E and D1n 2 ∈ E.
puting the payoff functions of two players, we have U1 = 
[u1 (B, D)]15×25 = G ⊗ (13 1T5 ) and U2 = [u2 (B, D)]15×25 = Theorem 5: Suppose that A5 holds and G3 is a circulant
GT ⊗ (13 1T5 ), where graph. Then, all topologies of the system are equilibrium topolo-
⎛ ⎞ gies.
2.5 4.5556 4.4444 3.9167 4.75 Proof: Let B1n 1 = ei and D1n 2 = ej . For the case of
⎜ 0.4444 0.3333 0.4444 0.4444 0.4167 ⎟ ei = ej , it is obvious from Properties 1 and 2 that u1 (B, D) =
⎜ ⎟
G=⎜
⎜ 0.5556 0.4815 0.4167 0.5556 0.5417 ⎟
⎟. u2 (B, D) = n23 . For the case of ei = ej , without loss of general-
⎝ 1.0833 1.0278 0.9444 0.7222 1.0312 ⎠ ity, we assume that i < j. Since G3 is a circulant graph, it follows
0.25 0.25 0.25 0.25 0.2 that L3 is a circulant matrix. Denote Qi = L3 + diag{ei } and
MA et al.: NASH EQUILIBRIUM TOPOLOGY OF MULTI-AGENT SYSTEMS WITH COMPETITIVE GROUPS 4965

obtained for Nash equilibrium. When the third group is a tree or

has a center vertex, interchangeable equilibrium solutions were
given. It was shown that the agents of third group might reach
consensus under the equilibrium topology, which is different
from previous theoretical results of containment control. In the
future, we may consider this game for multi-agent systems with
more than two competitive groups.

REFERENCES
[1] T. Vicsek, A. Czirók, E. Ben-Jacob, I. Cohen, and O. Schochet, “Novel
type of phase transitions in a system of self-driven particles,” Phys. Rev.
Lett., vol. 75, no. 6, pp. 1226–1229, 1995.
[2] Y. Tang, X. Xing, H. Karimi, L. Kocarev, and J. Kurths, “Tracking control
of networked multi-agent systems under new characterizations of impulses
and its applications in robotic systems,” IEEE Trans. Ind. Electron., vol. 63,
no. 2, pp. 1299–1307, Feb. 2016.
[3] D. Acemoglu, A. Ozdaglar, and A. ParandehGheibi, “Spread of (mis)
information in social networks,” Games Econ. Behav., vol. 70, no. 2,
Fig. 5. (a) Interaction graph G(B, D) and (b) state trajectories of all the pp. 194–227, 2010.
agents for Example 4. [4] S. Kang and H. Ahn, “Design and realization of distributed adaptive
formation control law for multi-agent systems with moving leader,” IEEE
Trans. Ind. Electron., vol. 63, no. 2, pp. 1268–1279, Feb. 2016.
Qj = L3 + diag{ej }. For a permutation matrix [5] S. D. J. McArthur et al., “Multi-agent systems for power engineer-
⎛ ⎞ ing applications—Part II: Technologies, standards, and tools for build-
Ii−1 0 0 ing multi-agent systems,”IEEE Trans. Power Syst., vol. 22, no. 4,
P =⎝ 0 0 In 3 +1−j ⎠ , pp. 1753–1759, Nov. 2007.
[6] V. Loia, V. Terzija, A. Vaccaro, and P. Wall, “An affine-arithmetic-based
0 Ij −i 0 consensus protocol for smart-grid computing in the presence of data un-
certainties,”IEEE Trans. Ind. Electron., vol. 62, no. 5, pp. 2973–2982,
we have Qi = P Qj P T . May 2015.
Noticing that P is orthogonal, we obtain that 1Tn (Qi )−1 ej = [7] Z. Zhang and M. Y. Chow, “Convergence analysis of the incremental cost
consensus algorithm under different communication network topologies
1n P (Qj )−1 P T ej .Because P T ej = ei and 1Tn P = 1Tn , we
T
in a smart grid,” IEEE Trans. Power Syst., vol. 27, no. 4, pp. 1761–1768,
know1Tn (Qi )−1 ej = 1Tn (Qj )−1 ei . Hence, from (25) and (26), Nov. 2012.
u1 (B, D) = u2 (B, D) = n23 . Consequently, we have S1∗ = S1 [8] A. Jabdabaie, J. Lin, and A. S. Morse, “Coordination of groups of mobile
autonomous agents using nearest neighbor rules,” IEEE Trans. Autom.
and S2∗ = S2 . In other words, graph G(B,D) is the equilibrium Control, vol. 48, no. 6, pp. 988–1001, Jun. 2003.
topology for every (B, D) ∈ S1 × S2 .  [9] D. Bauso, L. Giarré, and R. Pesenti, “Non-linear protocols for optimal
distributed consensus in networks of dynamic agents,” Syst. Control Lett.,
Example 4: The interaction graphs G1 , G2 , and G3 are de- vol. 55, no. 11, pp. 918–928, 2006.
picted in Fig. 5(a) where G3 is a circulant graph. It fol- [10] H. Liu, G. Xie, and L. Wang, “Containment of linear multi-agent systems
lows that S1 = {B = [bij ]5×3 , bij ∈ {0, 1}, 1T5 B13 = 1} and under general interaction topologies,” Syst. Control Lett., vol. 61, no. 4,
pp. 528–534, 2012.
S2 = {D = [dij ]5×2 , dij ∈ {0, 1}, 1T5 D12 = 1}. By comput- [11] Y. Zheng and L. Wang, “Containment control of heterogeneous multi-
ing the payoff functions of two players, we have U1 = agent systems,” Int. J. Control, vol. 87, no. 1, pp. 1–8, 2014.
[u1 (B, D)]15×10 = 2.5115 1T10 and U2 = [u2 (B, D)]15×10 = [12] L. Wang, F. Jiang, G. Xie, and Z. Ji, “Controllability of multi-agent sys-
tems based on agreement protocols,” Sci. China, Ser. F, vol. 52, no. 11,
U1 . Therefore, all strategy pairs are Nash equilibria, which il- pp. 2074–2088, 2009.
lustrate the effectiveness of theoretical results in Theorem 5. [13] Y. Guan, Z. Ji, L. Zhang, and L. Wang, “Decentralized stabilizability of
In particular, consider a strategy pair (B, D) whose interaction multi-agent systems under fixed and switching topologies,” Syst. Control
Lett., vol. 62, no. 5, pp. 438–446, 2013.
graph G(B,D) is shown in Fig. 5(a). The state trajectories of [14] G. Jing, Y. Zheng, and L. Wang, “Flocking of multi-agent systems with
agents are shown in Fig. 5(b). By Property 3, we know that if multiple groups,” Int. J. Control, vol. 87, no. 12, pp. 2573–2582, 2014.
two players choose same agents of G3 as their strategies, agents [15] E. Semsar-Kazerooni and K. Khorasani, “Multi-agent team cooperation:
A game theory approach,” Automatica, vol. 45, no. 10, pp. 2205–2213,
of G3 will reach consensus. In this example, two players choose 2009.
different agents, which gives rise to the result of not reaching [16] W. Ren and R.W. Beard, “Consensus seeking in multi-agent systems un-
consensus among agents of G3 . der dynamically changing interaction topologies,” IEEE Trans. Autom.
Control, vol. 50, no. 5, pp. 655–661, May 2005.
[17] L. Wang and F. Xiao, “Finite-time consensus problems for networks
V. CONCLUSION of dynamic agents,” IEEE Trans. Autom. Control, vol. 55, no. 4,
pp. 950–955, Apr. 2010.
In this paper, the agents of the multi-agent system were clas- [18] F. Xiao, L. Wang, and T. Chen, “Finite-time consensus in networks of
sified into three groups. Since the first and the second groups integrator-like dynamic agents with directional link failure,” IEEE Trans.
Autom. Control, vol. 59, no. 3, pp. 756–762, Mar. 2014.
could influence the third group, there exists competition between [19] Y. Cao and W. Ren, “Optimal linear-consensus algorithms: An LQR per-
them. We studied this competitive behavior in a noncoopera- spective,” IEEE Trans. Syst. Man, Cybern. B, Cybern., vol. 40, no. 3,
tive game theoretical framework. A two-player noncooperative pp. 819–830, Jun. 2010.
[20] J. Ma, Y. Zheng, and L. Wang, “LQR-based optimal topology of leader-
game was proposed, where a strategy pair corresponds an in- following consensus,” Int. J. Robust Nonlinear Control, vol. 25, no. 17,
teraction topology of the system. The necessary condition was pp. 3404–3421, 2015.
4966 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 64, NO. 6, JUNE 2017

[21] K. Hengster-Movric and F. Lewis, “Cooperative optimal control for multi- Jingying Ma was born in Ningxia, China. She
agent systems on directed graph topologies,” IEEE Trans. Autom. Control, received the Bachelor’s and Master’s degrees
vol. 59, no. 3, pp. 769–774, Mar. 2014. in applied mathematics from Tongji University,
[22] J. Qin, C. Yu, and B. D. O. Anderson, “On leaderless and leader- Shanghai, China, in 2002 and 2006, respec-
following consensus for interacting clusters of second-order multi-agent tively. She is currently working toward the Ph.D.
systems,”Automatica, vol. 74, pp. 214–221, 2016. degree in complex systems at Xidian University,
[23] Y. Zheng, Y. Zhu, and L. Wang, “Consensus of heterogeneous multi-agent Xi’an, China.
systems,” IET Control Theory Appl., vol. 16, no. 5, pp. 1881–1888, 2011. Since 2007, she has been a Teacher in the
[24] Y. Zheng and L. Wang, “Consensus of switched multi-agent systems,” School of Mathematics and Statistics, Ningxia
IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 63, no. 3, pp. 314–318, University, Yinchuan, China. Her research in-
Mar. 2016. terests include coordination of multi-agent sys-
[25] X. Lin and Y. Zheng, “Finite-time consensus of switched multiagent tems, game theory, and optimal control theory.
systems,” IEEE Trans. Syst., Man, Cybern., Syst., to be published, doi:
10.1109/TSMC.2016.2631659.
[26] M. Ji, G. Ferrari-Trecate, M. Egerstedt, and A. Buffa, “Containment con-
trol in mobile networks,” IEEE Trans. Autom. Control, vol. 53, no. 8,
pp. 1972–1975, Sep. 2008. Yuanshi Zheng was born in Jiangshan, China.
[27] G. Notarstefano, M. Egerstedt, and M. Haque, “Containment in leader- He received the Bachelor’s and Master’s de-
follower networks with switching communication topologies,” Automat- grees in applied mathematics from Ningxia Uni-
ica, vol. 47, no. 5, pp. 1035–1040, 2011. versity, Yinchuan, China, in 2006 and 2009, re-
[28] A.-H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and A. spectively, and the Ph.D. degree in control theory
Leon-Garcia, “Autonomous demand-side management based on game- from Xidian University, Xi’an, China.
theoretic energy consumption scheduling for the future smart grid,”IEEE He is currently with Xidian University. His re-
Trans. Smart Grid, vol. 1, no. 3, pp. 320–331, Dec. 2010. search interests include coordination of multi-
[29] D. Gu, “A differential game approach to formation control,” IEEE Trans. agent systems, consensus problems, contain-
Control Syst. Technol., vol. 16, no. 1, pp. 85–93, Jan. 2008. ment control, and coverage control.
[30] K. G. Vamvoudakis, F. L. Lewis, and G. R. Hudas, “Multi-agent differen-
tial graphical games: Online adaptive learning solution for synchronization
with optimality,” Automatica, vol. 48, no. 8, pp. 1598–1611, 2012.
[31] B. Gharesifard and J. Cortés, “Distributed convergence to Nash equi-
libria in two-network zero-sum games,” Automatica, vol. 49, no. 6, Long Wang was born in Xi’an, China, on Febru-
pp. 1683–1692, 2013. ary 13, 1964. He received the Ph.D. degree in
[32] A. Clark and R. Poovendran, “Maximizing influence in competitive envi- dynamics and control from Peking University,
ronments: A game-theoretic approach,” in Decision and Game Theory for Beijing, China, in 1992.
Security. Berlin, Germany: Springer, 2011, pp. 151–162. He held research positions at the University
[33] J. Qin, Q. Ma, H. Gao, Y. Shi, and Y. Kang, “On group synchronization of Toronto, Toronto, ON, Canada, and the Ger-
for interacting clusters of heterogeneous systems,” IEEE Trans. Cybern., man Aerospace Center, Munich, Germany. He
to be published, doi: 10.1109/TCYB.2016.2600753. is currently a Cheung Kong Chair Professor of
[34] J. Ma, Y. Zheng, and L. Wang, “Topology selection for multi-agent systems Dynamics and Control and the Director of the
with opposite leaders,” Syst. Control Lett., vol. 93, pp. 43–49, 2016. Center for Systems and Control at Peking Uni-
[35] J. Ma, Y. Zheng, B. Wu, and L. Wang, “Equilibrium topology of versity. He is also a Guest Professor at Wuhan
multi-agent systems with two leaders: A zero-sum game perspec- University, Wuhan, China, and Beihang University, Beijing, China. His
tive,”Automatica, vol. 73, pp. 200–206, 2016. research interests include complex networked systems, collective intel-
[36] C. D. Godsil, G. Royle, and C. D. Godsil, Algebraic Graph Theory. New ligence, and biomimetic robotics.
York, NY, USA: Springer, 2001. Prof. Wang serves as the Chairman of the Chinese Intelligent Net-
[37] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. Lon- worked Things Committee, and a Member of the IFAC Technical Com-
don, U.K.: Academic, 1995. mittee on Networked Systems. He is on the Editorial Boards of Science
[38] D. S. Bernstein, Matrix Mathematics: Theory, Facts, and Formulas. in China, Journal of Intelligent Systems, Journal of Control Theory and
Princeton, NJ, USA: Princeton Univ. Press, 2009. Applications, PLoS ONE, Journal of Intelligent and Robotic Systems,
and IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS.