A Simple Framework for Stability Analysis of State-Dependent Networks of Heterogeneous Agents

There are many motivating examples of relationships in political, social, and engineering applications, which are governed by complex networks of heterogeneous agents. Agents may possibly belong to multiple groups and be connected by multi-layered networks. The networks can also be dynamic in the sense that they can vary over time depending on the agents’ status. As a few illustrative examples one can consider (Figure 1):

• •

  In social networks, there are often clear affinities among people based on shared political or cultural beliefs. However, on specific issues, alliances form among people from different groups. Almost every congressional vote provides an example of this phenomenon, where some representatives break away from their respective parties to vote with the other party.

• •

  In formation control a basic task is to design a distributed protocol so that a set of robots collectively form a certain structure. Robots have different communication capabilities and can only communicate with those in their local neighborhood. Consequently, the communication network between them may vary depending on their relative distances from each other.

• •

  Relationships between countries in the Middle East, and their ties to the US and Russia are nuanced, with affiliations changing over time, and depending on context. Similarly, relationships among terrorist organizations, such as ISIS, Al Qaeda, Taliban, and LeT, often change due to their battle for supremacy, and their fight for the allegiance of their extremist followers.

• •

  In opinion systems such as political election polls, individuals initially have different opinions about a certain topic/candidate. Individuals frequently interact and become friend/unfriend depending on how close their opinions are from each other. In particular, through such interactions their opinions gradually form and a collective opinion eventually emerges.

Motivated by the above, and many other similar examples, our objective in this work is to provide a simple framework to understand and analyze the behavior of networks of heterogeneous agents with a rich dynamic network structure which may evolve or vary based on agents’ states. To this end, we provide new connections between analysis of multi-agent network systems and some developed methods in the mature fields of successive optimization and game theory. Utilizing such connections, we establish Lyapunov stability and convergence of several classes of heterogeneous multi-agent network systems with switching state-dependent dynamics.

There has been a rich body of literature on analysis of distributed multi-agent network systems, mainly from the static point of view, in which a set of agents iteratively interact over a fixed communication network so as to achieve a certain goal such as consensus or optimizing a global objective function. The classical models of Degroot [1] and Friedkin& Johnsen [2] in social science are two special types of such systems. As the literature on this area is quite vast, we refer interested readers for an overview to [3] and [4]. However, a crucial assumption in almost all of these works is that the communication network among the agents is fixed. Often these results can be generalized to time-varying networks by assuming a certain “independency” between the network process and the state dynamics. For instance, one of the commonly used assumptions is that the network dynamics are governed by an exogenous process which is uncoupled from the state dynamics [5, 6, 7, 8, 9, 10]. A generalization of this idea is to consider multi-agent network dynamics where the network and state dynamics are allowed to be coupled, however a certain condition such as network connectivity or communication symmetry must be satisfied at each time instance [11, 12, 13, 14]. A further extension of these results is the line of works in [15, 16] which shows that any sequence of stochastic matrices viewed as update matrices of multi-agent network dynamics admits a “Lyapunov” type function. Unfortunately, such Lyapunov functions are based on another so-called adjoint dynamics which depend on the future instances of the original dynamics. Thus, unless there is a strong inherent property in the underlying stochastic matrices [17, 14], it seems unlikely to leverage the co-evolution of original and adjoint dynamics so as to establish meaningful convergence results.

While the above techniques can properly address a large class of multi-agent network systems, there are still many examples which do not fit into any of the aforementioned frameworks (or the application of the above techniques provides very poor results on the behavior of the multi-agent system). One of the main reasons is that most of the developed frameworks for analysis of multi-agent network systems aim to isolate the evolution of network dynamics from those of state dynamics in which case one derives a conclusion about local time instances (e.g. one time slot or a window of time instances) and then generalizes this behavior to the entire trajectories. But the main question here is that what if the network and state dynamics are completely determined endogenously so that no local property can be assumed or checked a priori? In other words, if we do not take into account the actual correlation between network and state dynamics (or how they evolve in terms of each other), it seems hopeless to have a good understanding of the overall behavior of the dynamics. This shortcoming is even more pronounced once we take into account agents heterogeneity or asymmetric communication among them. In this work we show that despite these challenges it is still possible to capture the co-evolution of network-state dynamics for several (and perhaps many other) multi-agent network dynamics even under asymmetric or heterogeneous environment. While the evolution of state dynamics are commonly captured by difference/differential equations, one of the key novelties of our work is to explicitly derive analogous equations for network dynamics by explicitly introducing network variables and couple them with the state variables. This allows us to derive network topology from state dynamics and vice versa and even incorporate other constraints into the system behavior.

The rest of the paper is organized as follows: In Section II we provide a general framework which we use frequently throughout the paper. In Section III we apply this framework to a well-known state-dependent switching dynamics from social sciences, known as Hegselmann-Krause (HK) opinion dynamics, and extend it to an edge-heterogeneous setting with additional communication constraints. In particular, we show how this approach can handle up to some extent asymmetric communication among the agents in the HK model. In section IV we introduce a new class of asymmetric opinion dynamics, namely nearest neighbor dynamics, and show that our framework can also be applied suitably to these models despite the fact that the underlying communication networks suffer from asymmetry. In Section V, we provide an application of our framework to game-theoretic settings and show how the existing results in game theory can be leveraged to analyze heterogeneous multi-agent network dynamics. We conclude the paper by identifying some future directions of research in Section VI.

We adopt the following notations throughput the paper: We use bold symbols for vectors. Given a vector $\boldsymbol{v}$ we let $diag(\boldsymbol{v})$ be a diagonal matrix with vector $\boldsymbol{v}$ as its diagonal elements and zero, everywhere else. Moreover, we denote the transpose of $\boldsymbol{v}$ by $\boldsymbol{v}^{T}$. For a positive integer $n\in\mathbb{Z}^{+}$ we set $[n]:=\{1,2,\ldots,n\}$. We let $\boldsymbol{1}$ be a column vector of all ones. Given a positive-definite matrix $Q$ and a vector $\boldsymbol{v}\in\mathbb{R}^{n}$, we let $\|\boldsymbol{v}\|^{2}_{Q}=\boldsymbol{v}^{T}Q\boldsymbol{v}$, and $\|\boldsymbol{v}\|$ to be the Euclidean norm of $\boldsymbol{v}$. Given a real number $x\in\mathbb{R}$ we set $(x)^{-}:=\min\{0,x\}$. We denote the cardinality of a finite set $S$ by $|S|$.

Let us first focus on the homogeneous HK model. Although convergence and detailed analysis of the homogeneous HK model have been established and studied extensively in the past literature (see, e.g., [25] for a comprehensive survey), in this subsection we provide a simple argument to show why this model fits into our framework. This will allow us to generalize stability of homogeneous HK model to account for higher degrees of heterogeneity or asymmetry among the agents.

Let us define $\mathcal{G}_{t}=([n],\mathcal{E}_{t})$ to be the communication graph at a generic time $t$ such that there is an edge between agents $i$ and $j$ at time $t$, i.e., $(i,j)\in\mathcal{E}_{t}$ if and only if $j\in N_{i}(t)$. Note that in the homogeneous HK model the communication graph is undirected as if agent $j$ is a neighbor of agent $i$, the converse is also true. In fact, what makes the analysis of HK dynamics challenging is the strong coupling between the evolution of the network $\mathcal{G}_{t}$ and the state $\boldsymbol{x}(t)$. This is because at any time $t$ the state vector $\boldsymbol{x}(t)$ determines the network topology $\mathcal{G}_{t}$, and this new network determines the state vector at the next time step $\boldsymbol{x}(t+1)$. This puts HK dynamics to the class of complex time-dependent and state-dependent network dynamics [35, 36, 37, 38, 39, 40]. In particular, the communication network $\mathcal{G}_{t}$ may switch many times depending on how the opinion vectors evolve which brings additional complication to the analysis.

Now let us consider the following objective function comprised of two block variables, namely $\boldsymbol{y}\in\mathbb{R}^{n}$ and $\boldsymbol{\lambda}=(\lambda_{ij})\in[0,1]^{n\times n}$,

This function can also be written in a compact form as $f(\boldsymbol{y},\boldsymbol{\lambda})=\boldsymbol{y}^{T}\mathcal{L}\boldsymbol{y}-tr(\mathcal{L})\epsilon^{2}$, where $\mathcal{L}:=diag(\boldsymbol{\lambda}\boldsymbol{1})-\boldsymbol{\lambda}$, and $tr(\cdot)$ denotes the trace function. Intuitively, the block variable $\boldsymbol{\lambda}$ is meant to capture the communication network $\mathcal{G}_{t}$, and the block variable $\boldsymbol{y}$ captures the opinion states. Note that if we restrict $\lambda_{ij}$s to binary variables in $\{0,1\}$, then $\boldsymbol{\lambda}$ simply represents the adjacency matrix of a network of $n$ agents where $\lambda_{ij}=1$ if there is a directed edge $(i,j)$ from node $i$ to node $j$, and $\lambda_{ij}=0$ otherwise. Moreover, for such a binary block variable $\boldsymbol{\lambda}$, the matrix $\mathcal{L}$ is precisely the Laplacian matrix of the communication network associated with $\boldsymbol{\lambda}$. Although we still need to assume that $\boldsymbol{\lambda}\in\{0,1\}^{n\times n}$, to avoid complication of handling integral variables, for now we allow $\lambda_{ij}$s to vary continuously in the interval $[0,1]$. As we shall see soon the integrality of network variables will be automatically achieved during iterations of the BCD method.

Now let us consider the BCD method applied to the objective function (12) with block variables $\boldsymbol{x}$ and $\boldsymbol{\lambda}$. For a generic time $t$, let us fix the state variable to $\boldsymbol{y}=\boldsymbol{x}(t)$. Minimizing (12) with respect to the network variable $\boldsymbol{\lambda}\in\Lambda=[0,1]^{n\times n}$ we obtain,

where

This simply follows because the objective function $f(\boldsymbol{x}(t),\boldsymbol{\lambda})$ is a linear function of the network variable $\boldsymbol{\lambda}$ and achieves its minimum in an extreme point of $[0,1]^{n\times n}$. In particular, the optimal extreme point can be found by an easy inspection as given in (14). But note that $\boldsymbol{\lambda}_{t}$ is precisely the adjacency matrix of the communication graph $\mathcal{G}_{t}$ in the homogeneous HK model (recall that in the homogeneous HK model two agents are each others’ neighbors at time $t$ if and only if their distance is at most $\epsilon$). Therefore, fixing the state variable to $\boldsymbol{x}(t)$ and minimizing (12) with respect to the network variable exactly delivers the adjacency matrix of the communication network $\mathcal{G}_{t}$ in the homogeneous HK model at that time.

Now let us fix the network variable to the minimizer $\boldsymbol{\lambda}_{t}$ which represents an undirected graph with associated Laplacian matrix $\mathcal{L}_{t}$. It is well-known that given a fixed undirected graph with Laplacian matrix $\mathcal{L}_{t}$ and arbitrary values $\{y_{i},i\in[n]\}$ at its $n$ verticies, the quadratic function $\Phi(\boldsymbol{y},\boldsymbol{\lambda}_{t}):=\boldsymbol{y}^{T}\mathcal{L}_{t}\boldsymbol{y}$ is nonincreasing if each node updates its value to the average value of its neighbors [41, 42]. In other words, defining $\boldsymbol{y}^{\prime}=A_{t}\boldsymbol{y}$ where $A_{t}:=(diag(\boldsymbol{\lambda}_{t}\boldsymbol{1}))^{-1}\boldsymbol{\lambda}_{t}$, we have $(\boldsymbol{y}^{\prime})^{T}\mathcal{L}_{t}\boldsymbol{y}^{\prime}\leq\boldsymbol{y}\mathcal{L}_{t}\boldsymbol{y}$. But note that $A_{t}$ is precisely the update matrix of the homogeneous HK model given in (10), which means that for the specific choice of $\boldsymbol{y}=\boldsymbol{x}(t)$ we have $\boldsymbol{y}^{\prime}=\boldsymbol{x}(t+1)$. Thus the evolution of the homogeneous HK dynamics is governed by the application of BCD method to the objective function (12). Appealing to Proposition 1, this shows that $f(\boldsymbol{x}(t+1),\boldsymbol{\lambda}(t+1))\leq f(\boldsymbol{x}(t),\boldsymbol{\lambda}(t))$, implying that homogeneous HK dynamics admit a Lyapunov function.

In this part we show how the framework of Section II can extend the analysis of homogeneous HK model by capturing higher degrees of heterogeneity or constraints. For this purpose, we consider restricted edge-heterogeneous HK model which is a variant of the homogeneous HK model with the following two additional changes:

1) Edge Heterogeneity: Consider the same dynamical system as in the homogeneous HK model (10), except that the distance between every pair of nodes is measured based on possibly a different confidence bound. More precisely, let $\{\epsilon_{ij}=\epsilon_{ji}>0,\forall i\neq j\}$ be a set of fixed thresholds (one for each pair of agents) so that agents $i$ and $j$ at time instance $t$ can communicate if and only if their distance at that time is less than $\epsilon_{ij}$, and thus $N_{i}(t)=\{j\in[n]:|x_{i}(t)-x_{j}(t)|\leq\epsilon_{ij}\}$. This captures the heterogeneous relationship among individuals due to family or other social ties. For instance two family members will still continue to communicate even if their opinions are relatively far from each other, while two strangers are more likely to terminate their interactions as soon as their opinions slightly deviate from each other. As before we assume that at each time instance agents update their opinions by taking the arithmetic average of their neighbors’ opinions, determined based on the heterogeneous thresholds $\epsilon_{ij}$. Note that the homogeneous HK dynamics is a special case of the edge heterogeneous setting where $\epsilon_{ij}=\epsilon,\forall i\neq j$.

2) Communication Restrictions: Other than closeness in opinion, often there are other important factors which determine whether or not two agents should communicate. For instance, individuals often communicate with those who have closer opinion to them and are within small geographic distance from them. One direct way of handling such restriction is to add an extra component to each agent’s opinion where this new component encodes the geographic location of that agent. As a result, in this higher dimensional opinion space two agents are each others’ neighbors if each component of their opinion vectors (and in particular the geographic component of their opinion vector) are close to each other. This implies that two agents are eligible to communicate if they are close to each other both opinionwise and geographic-wise.³³ 3 Although this approach increases the dimension of the opinion space, yet most of the results such as convergence of the dynamics can be extended to this higher dimensional setting [43]. An alternative approach however for imposing new constraints is to consider a predefined underlying network $\mathcal{G}$ which restricts the agents’ interactions to only those who are both connected through the edges of $\mathcal{G}$ and have closer opinion to each other. Intuitively, one can imagine running HK dynamics over the graph $\mathcal{G}$. Thus, denoting the communication network of the edge-heterogeneous HK dynamics at time $t$ by $\mathcal{G}(t)$, the actual communication graph at time $t$ is given by the intersection of edges which appear in both $\mathcal{G}$ and $\mathcal{G}(t)$. This second approach is particularly more suitable when there are certain hard communication constraints among individuals due to age gaps, gender restrictions, or other social laws.

Finally, in this subsection we take one step further and consider a special case of the node-heterogeneous HK model. As we mentioned earlier, the dynamics of node-heterogeneous HK model follow exactly the same update rule as homogeneous HK model given in (10) except that different agents might have different confidence bounds $\epsilon_{i},i\in[n]$. Unfortunately, up to the time of writing this paper there is no general result which either proves or disproves convergence of the node-heterogeneous HK model (although some partial results concerning stability of these dynamics are known [44, 43, 32]). In particular, in the recent work [35], the authors have used an algorithmic approach to show convergence of a special case of the node-heterogeneous HK model, namely 0-1 HK model, in which the confidence bound of each node is restricted to be either 0 or $\epsilon=1$ (i.e., $\epsilon_{i}\in\{0,1\},\forall i\in[n]$). It is worth noting that due to heterogeneous confidence bounds, the communication network in the 0-1 HK model is no longer undirected (symmetric). While the proof in [35] is fairly long and algorithmic, here we provide a simple argument based on the BCD framework to establish Lyapunov stability of the 0-1 HK dynamics. An important advantage of our approach is that i) it provides an improved Lyapunov drift which can be useful towards convergence rate analysis, and ii) it provides a clear explanation of why the asymmetric 0-1 HK dynamics can still be treated as the symmetric homogeneous HK model.

Consider a set of $[n]$ agents where the opinion of agent $i\in[n]$ at time $t=0,1,2,\ldots$ is given by a vector $x_{i}(t)\in\mathbb{R}^{d}$. At each iteration $t$ one agent $i\in[n]$ is selected based on some selection rule (e.g. uniformly at random) and updates its opinion at the next time step to $x_{i}(t+1)=\mu_{i}x_{i}(t)+(1-\mu_{i})x_{r(i)}(t)$, where here $r(i):=\arg\min_{j\in[n]\setminus\{i\}}\|x_{i}(t)-x_{j}(t)\|$ denotes the closest agent to $i$ with respect to profile $\boldsymbol{x}(t)\in\mathbb{R}^{n\times d}$, and $\mu_{i}\in(0,1)$ is an agent-specific parameter. For all other agents $j\neq i$, we set $x_{j}(t+1)=x_{j}(t)$.

The rationale behind introducing nearest neighbor dynamics is that often individuals get influenced by their closest friend/partner/leader so that depending on their stubbornness (captured by $\mu_{i}$) they are willing to compromise in order to get closer to their friend/partner/leader. It is important to note that the communication network in the nearest neighbor dynamics is asymmetric so that if $r(i)$ is the closest agent to $i$, it does not imply that $i$ is also the closest agent to $r(i)$ (i.e., in general $r(r(i))\neq i$).⁵⁵ 5 In fact, one can show that the communication network at each time instance is comprised of disjoint directed trees where the out-degree of each node is equal to 1. Moreover, the communication network which determines the “closest relationships” evolves as a function of agents’ opinions which can switch many times based on trajectory of the dynamics. Nevertheless, as we shall see in Theorem 4 such heterogeneous asymmetric state-dependent dynamics will converge to an equilibrium point as defined below:

It is worth noting that the definition of $\epsilon$-equilibrium implies that each agent will lie within a distance of at most $\epsilon n$ from its limit point. This is because if the communication network in an $\epsilon$-equilibrium is connected, the maximum distance between every two nodes is at most $\epsilon n$ so that the convex hull of all the opinions can have diameter at most $\epsilon n$. Since the nearest neighbor dynamics evolve inside of this convex hull for all the future iterations, the limit point (if it exists) will also lie in this convex hull. Similarly, if an $\epsilon$-equilibrium contains more than one connected component, then either the distance between their convex hulls is less than $\epsilon$, in which case a similar argument as above for the convex hull of the union of those components can be applied, or the distance between those components is more than $\epsilon$, in which case those components evolve separately from each other so that the limit points of each component lie within its own convex hull (and again the above argument applies).