Machine learning of percolation models using graph convolutional neural networks

Hua Tian,1 Lirong Zhang,1 Youjin Deng,3, 4, 5, ∗ and Wanzhou Zhang2, 3, 6, †

1
College of Information and Computer, Taiyuan University of Technology, Taiyuan 030024, China
2
College of Physics, Taiyuan University of Technology, Shanxi 030024, China
3
Hefei National Laboratory for Physical Sciences at the Microscale and Department of Modern Physics,
University of Science and Technology of China, Hefei 230026, China
4
Shanghai Research Center for Quantum Sciences, Shanghai 201315, China
5
MinJiang Collaborative Center for Theoretical Physics,
College of Physics and Electronic Information Engineering, Minjiang University, Fuzhou 350108, China
6
CAS Key Laboratory for Theoretical Physics, Institute of Theoretical Physics,
Chinese Academy of Sciences, Beijing 100190, China
arXiv:2207.03368v1 [cond-mat.stat-mech] 7 Jul 2022

(Dated: July 8, 2022)

Percolation is an important topic in climate, physics, materials science, epidemiology, finance,
and so on. Prediction of percolation thresholds with machine learning methods remains challenging.
In this paper, we build a powerful graph convolutional neural network to study the percolation in
both supervised and unsupervised ways. From a supervised learning perspective, the graph neural
network simultaneously and correctly trains data of different lattice types, such as the square and
triangular lattices. For the unsupervised perspective, combining the graph convolutional neural
network and the confusion method, the percolation threshold can be obtained by the “W” shaped
performance. The finding of this work opens up the possibility of building a more general framework
that can probe the percolation-related phenomenon.

Introduction.— More than half a century ago, S.R.

Broadbent and J.M. Hammersley proposed the percola-
tion model[1]. It can help one understand physics [2],
materials science [3], complex networks [4, 5], epidemi-
ology [6], geology climate [7], quantum computation [8],
and other fields [9, 10]. Many classical statistical physics
models can be mapped to the correlated percolation mod-
els [11–15] in geometric perspectives. In 2016, M. Roger
et al. and M. Troyer et al. successfully used neural (a) (b)
networks (NN) to identify the different phases of both
classical and quantum statistical matter[16, 17]. Since FIG. 1. An occupied state for the site percolation model
then, supervised machine learning (ML) methods have is not percolating for the square lattice (a), but it can be
also been used to study a large number of phase transi- percolating for the triangular lattice (b).
tions [18]. Meanwhile, attempts are being made to study
the phase transition using unsupervised ML methods.
These methods include principal component analysis between sites is an important factor in determining the
methods [19], t-distributed stochastic neighbor embed- percolation threshold and must be encoded in the train-
ding methods [20], diffusion mapping methods [21, 22], ing set. Otherwise, the NN is fed only the occupied states
the confusion scheme [23] and auto-encoders [24]. of sites and then cannot distinguish the different lattice
For the percolation model, the supervised learning is structures. In addition, there is a spanning cluster in the
able to predict the percolation threshold and determine system when percolation occurs, and percolation is a non-
critical exponents [25, 26]. However, by entering the oc- local feature that is difficult to be captured by a NN [30].
cupied states of the site percolation, the NN cannot dis- This problem may seem to simple, but the percolation
tinguish the lattice types that the configurations come model has many important applications in various fields
from. In addition, for unsupervised learning, despite a and the solution to this problem is non-trivial.
great success application in correlated systems, such as The previous work [30] modified the input datasets
the Ising model, however, it can not detect the perco- of percolation configurations with well-characterized
lation thresholds properly [27–30]. Specifically, an occu- datasets, i.e., the diluted Ising model. In this paper,
pied state for the site percolation model is not percolating we choose another option, modifying not the dataset but
for the square lattice (see Fig. 1 (a)), but it can be perco- the machine learning framework, that is, turning the NN
lating for the triangular lattice ( see Fig. 1 (b)). A neural into a more powerful graph convolutional neural network
network (NN) is unable to distinguish between these two (GCN) [31]. GCN is a deep learning-based scheme that
cases. operates on a graph, in such a way that the graph encodes
For the site percolation, the connection information both the state of the site and the connection information
 2

𝑒𝑒12 𝑒𝑒23
𝑣𝑣1 𝑒𝑒21 𝑣𝑣2𝑒𝑒32 𝑣𝑣3 𝑣𝑣𝑖𝑖 ℎ𝑖𝑖0
𝑒𝑒14 𝑒𝑒25 𝑒𝑒36
𝑒𝑒41 𝑒𝑒45 𝑒𝑒52 𝑒𝑒56 𝑒𝑒63
𝑙𝑙
� 𝜎𝜎 ⋅ ⊙ℎ𝑔𝑔(⋅)
𝑣𝑣4 𝑒𝑒54 𝑣𝑣5 𝑒𝑒65 𝑣𝑣6 𝑖𝑖
𝑒𝑒47 𝑒𝑒58 𝑒𝑒69
𝑒𝑒74 𝑒𝑒78 𝑒𝑒85 𝑒𝑒89 𝑒𝑒96 𝑒𝑒𝑖𝑖𝑖𝑖
𝑣𝑣7 𝑒𝑒87 𝑣𝑣8 𝑒𝑒98 𝑣𝑣9

(𝑎𝑎1 ) Square lattice (𝑎𝑎2 ) Graph representation (𝑎𝑎3 ) Embedding (𝑏𝑏1 ) Convolution layer

𝑒𝑒12 𝑒𝑒23
output

𝑒𝑒12 ℎ1 𝑒𝑒21 ℎ2 𝑒𝑒32 ℎ3

𝑝𝑝𝑐𝑐 𝑥𝑥1 𝑒𝑒21
𝑒𝑒14 𝑒𝑒25 𝑒𝑒36
𝑒𝑒13 𝑥𝑥 𝑒𝑒41 𝑒𝑒45 𝑒𝑒52 𝑒𝑒56 𝑒𝑒63
p 𝑒𝑒24 2
𝐺𝐺 𝑒𝑒31 𝑒𝑒43 𝑒𝑒42 ℎ4 𝑒𝑒54 ℎ5 𝑒𝑒65 ℎ6
𝑒𝑒47 𝑒𝑒58 𝑒𝑒69
performance

𝑥𝑥3 𝑒𝑒34 𝑥𝑥4 𝑒𝑒74 𝑒𝑒78 𝑒𝑒85 𝑒𝑒89 𝑒𝑒96

ℎ7 𝑒𝑒87 ℎ8 𝑒𝑒98 ℎ9
𝑝𝑝𝑐𝑐
p
(𝑐𝑐2 ) Output curve (𝑐𝑐1 ) Full connection layer (𝑏𝑏2 ) Pool layer

FIG. 2. The main idea of this work. (a) The three snapshots, graph representation containing sites {vi } and {eij } and the
embedding layer, (b) are the convolution and pooling layers, (c) are the full connection layer and the ouput layer. Supervised
learning results in an ‘X’-shaped curve and unsupervised learning corresponds to a ‘W’-shaped neural network performance.

between the sites. configurations and represent them as directed graphs.

The GCN performs a convolution operation with the For site-percolation on a square lattice, the configura-
feature (usually a vector) of the surrounding nodes and tion is obtained via randomly occupying sites according
the edges between the nodes, and then iteratively up- to a certain probability pi ∈ [pmin , pmax ]. Three snap-
dates the feature information of each node. After l times shots are shown with the occupation probabilities p =
of convolution, each node contains the feature of its l- 0.2, 0.593 (percolation threshold) and 0.8 respectively.
order neighboring nodes, and the local features of the In the configuration, the sites are denoted as the nodes
configuration are extracted. {vi }, and the connections between sites are denoted as
The main highlights of this article are the following directed edges {evi ,vj }, where i and j are the indexes of
two points. (I) For supervised learning, we can build the sites. The feature of evi ,vj can be marked by attrvi ,vj
a more general framework, that can simultaneously de- and it is assigned to 1, when the node vi and node vj are
termine percolation thresholds for different lattices, such occupied at the same time; on the contrary, attrvi ,vj is
as triangular lattices and square lattices. Datasets with assigned to 0. Figure 2 (a3) describes an embedding im-
different lattice structures can participate in the train- plemented by a small two-layer fully connected network.
ing simultaneously. Extending to other lattices is also In the first layer, there are 2 neurons. The first neu-
straightforward and feasible. (II) For unsupervised learn- ron indicates the occupation status of the i-th node and
ing, information about connections between sites is fed to the second indicates the occupation probability. These 2
the GCN and correlations between sites are captured to neurons are then mapped to the second layer with more
a large extent. Using the unsupervised method [23], we neurons. Their values are labeled as h0i (a vector), and
can get a percolation threshold that cannot be obtained are prepared for the next layer of graph convolution. The
by the NN unsupervised method [30]. In addition, tech- embedding is done in order to allow better performance
nologically, we use state-of-the-art graph convolution and of our training because the richer features are considered.
soft allocation of learnable pooling [32] to build our GCN.
The basic framework of the method.— Figure 2 shows (b) Convolution and pooling. In Fig. 2 (b1), the con-
the training process of a batch, consisting of the following volution is performed and the basic idea is from Ref. [33].
three steps, (a) data input, (b) graph convolution and GCN learns the features from neighboring nodes through
pooling and (c) output, respectively. the edges. The features of each node, hli , are updated by
(l−1)
(a) Data input. In Figs. 2 (a1)-(a2), we generate the convolution of the features of neighboring nodes hj
 3

1 2 3 4 5 and probabilistic matrix S ∈ Rnk ×nk+1 is used and given

3,7,9,13
by
8 6 7 8 9 10   
7, 9, S (k) = softmax GCNConvpooling A(k) , X (k) , (2)
11, 15, 11 13 14 15
12 13 14 12
13, 15,
17 19
where X (k) is the feature vector of the super node in
18 16 17 18 19 20
the k-th pooling layer. The operator GCNConv used
13,17,19,23 in pooling is from Ref. [34], its specific definition is as
21 22 23 24 25
follows,
(a) (b) 1
GCNConvpooling : X k+1 = D̂ 2 ÂD̂− 2 X k Θ,
1
(3)

FIG. 3. After two graph convolutions, site 13 obtains infor- where  = A + I denotes the adjacency matrixPwith in-
mation from its next nearest neighbor site. (a) the l = 1-th serted self-loops for the super-nodes and D̂ii = j=0 Âij
iteration (b) the l = 2-th iteration. The square lattice with is diagonal degree matrix. Θ is the filter parameter ma-
L = 5 is used. After l iterations, the site 13 gets the informa- trix representing the probability that the node is assigned
tion of l-order neighbors. to any clusters (super nodes) for the next hierarchical
pooling layer. Then, one has to calculate the feature
X (k+1) and the adjacency matrix A(k+1) of the next layer,
and itself iteratively. In Fig. 2 (b2), the pooling is per-
by the equations as follows,
formed [32]. From a 3 × 3 graph, one obtains a super
node, representing the feature of the whole graph. T
X (k+1) = S (k) X (k) ∈ Rnk+1 ×d , (4)
(c) Output. In Fig. 2(c1), the final classification re-
(k) T
sult is obtained by training the full connection neural A(k+1) = S A(k) S (k) ∈ Rnk+1 ×nk+1 . (5)
network. In Fig. 2(c2), the results of the output neuron
have an “X” shape output for supervised learning. For In general, the iteration obeys the flow as in equation
GCN
the unsupervised method, one determines the true pc by A(k) , X (k) −→ S (k) −→ A(k+1) , X (k+1) .
the position of the peak of “W” shaped performance [23]. Message passing.— The previous operations of convo-
The algorithm for a batch of training is shown in ap- lution and pooling are implemented based on the message
pendix I. passing mechanism [35], by which one can obtain infor-
Convolution and pooling in more detail.— In each con- mation from “adjacent” nodes, and realize the “convolu-
volution layer, the features on the neighboring nodes and tion” operation on the graph and aggregate the informa-
edges are concatenated together, i.e., zijl
= hlvi ⊕ hlvj ⊕ tion of surrounding nodes. The message passing consists
attrvi ,vj , where ⊕ represents the concatenation opera- of the delivery and the readout steps. During the delivery
tion. Then, the convolution is performed by [33], step, the feature hlvi of each node vi is updated according
to Eq. 6 and Eq. 7 given by,
X
mlvi = l
σ(zij Wfl + blf )  g(zij
l
Wsl + bls ), (1)
vj ∈N (vil )
X
mlvi = Ml (hl−1 l−1
vi , hvj , evi,j ), (6)
vj ∈N (vil )
where {vj |vj ∈ N (vi )} represents the collection of adja-
cent nodes of node vi , W is the weight matrix and b is the hlvi = Ul (hl−1 l
vi , mvi ), (7)
bias, l ∈ {1, 2, 3, · · · } represents the convolution of layer
number, σ (·) is the sigmoid function σ (x) = 1+e1−x , g (·) where Ml (·) is the aggregate function, and Ul (·) is the
is the softplus function g (x) = log (1 + ex ). Finally, the update function. In Figs. 3 (a) and (b), when l = 1,
new feature hl+1 l+1 l l the site 13 absorbs messages from its four neighbors 12,
vi is updated by hvi = hvi + mvi .
In the pooling layer, we use a differentiable graph pool- 14, 8, and 18. At the same time, these neighbors also
ing module [32], based on the graph hierarchical pooling. absorb information from their respective neighborhoods,
In Figs. 2 (b2), in a specified graph collapse way, a 3 × 3 i.e., h0i is replaced with h1i . For the second iteration,
graph becomes a new graph with 2 × 2 nodes and finally l = 2, site 13 indirectly grabs information from the next
collapses to a super node. The way of graph collapse can nearest neighbors, connected by the dashed blue lines. In
be realized by a cluster assignment matrix S ∈ Rn1 ×n2 , the readout stage, the function R(·) is used to calculate
where n1 is the number of nodes in the graph Gl and the feature vector of the whole graph as follows,
n2 is the number of nodes of the new graph. Si,j = 1 y = R({hlv |v ∈ V (G)}), (8)
indicates that the node vi belongs to the j-th cluster or
supernode. Another example is shown in the appendix II, where R(·) sums features from all nodes and it can be
where the matrix S is defined manually (hard). In the some learnable differentiable function, for example, the
real simulation, for a graph with nk nodes, a soft learn pooling function.
 4

Confusion method.— For the percolation controlled (a) 1.0

by p, the configurations represented in graphs can be

classified into percolating at p ≥ pc and non percolating 0.8 SL8 p
p ≤ pc . Therefore, if one separates the data with different SL12 0.6
c

p into two classes, the separation according to pc results

output
SL16
0.6
SL20
in the largest differences between the two classes, there-
TL8 0.5
fore leads to a high classification accuracy obtained by 0.4 TL12 0 1/20 1/8

our GCN. The idea of the confusion method [23] here is TL16
1/L

to find such performance peaks, which is helpful to detect 0.2

TL20

the percolation threshold. To perform this method, one

has to prepare a dataset first. We divide the parameter 0.0
range [pmin , pmax ] into Np parameters {pi } and generate 0.4 0.5 0.6 0.7
Ns configurations at each parameter pi . This gives us to- p

tally N = Np × Ns configurations. Then, one can make

the trial labels and perform training and testing. The pi (b)

is chosen as the trial threshold for generating trial labels,

and all graphs generated with pj < pi are specified as the
label 0, and the rest are specified as label 1. One repeats
the previous two operations with all trial thresholds {pi }
in the range [pmin , pmax ], and would output the perfor-
mance curve as expected. A binary classification GCN
as shown in Fig. 2.
Results.— Our first idea is to build a general GCN
that can predict the percolation thresholds of different Sq L=8

Sq L=16
lattices simultaneously, i.e., we train the GCN using the
input graphs with a fixed number of nodes but different
numbers of edges simultaneously, which is impossible for
a NN. To test and validate the idea, we first study site
percolation models on the triangular and square lattices FIG. 4. (a) Supervised learning outputs double-“X” shaped
using supervised machine learning methods. curves with system sizes L = 8, 12, 16, 20. “SL” and “TL”
The results are shown in Fig. 4 (a). We see the two “X” means the square lattices and the triangular lattices, respec-
shaped outputs for triangular lattices (TL) and square tively. Inset is a finite size scaling for the pc . (b) The un-
lattices (SL). The lattice sizes are L = 8, 12, 16, 20, re- supervised leasing produces “W”-shaped performance curves
for system sizes of L = 8, 16, respectively.
spectively. In the training dataset, 100 configurations are
generated for each lattice type (SL, TL) under each pi .
The data set of different lattice types but the same size
also put degree centrality in the feature for each node in
of configurations are fed into the same neural network to
the embedding, which is defined as
participate in training. The same network, however, can
predict the thresholds on different types of lattices simul- degree(vi )
taneously, which is impossible to achieve with non-graph dc = , (9)
(ntotal − 1)
networks. The obtained threshold is 0.488(2) for the TL
and 0.58(1) for the SL by the inset in Fig. 4 (a). The where degree(vi ) is the degree of the node vi , ntotal is the
accuracy can continue to be improved by better training total number of nodes. The features such as properties
of the neural network. We also test the bond percolation of edges, occupation probability, and degree centrality
and the predictions are also acceptable, but not shown. make the network perform better.
The second idea is to predict the percolation threshold Conclusion and Outlook.— In conclusion, using the ad-
using unsupervised learning. In Fig. 4(b), using the con- vantages of graph neural networks, we have solved a dif-
fusion method [23, 29, 30], we obtain the performance ficult problem since the year 2017 [23, 25–27, 29, 30].
with the “W” shape. In these datasets, there are 100 The NN can not distinguish the different structures of
-1000 configurations per pi with pi ∈ [0, 1] interval of lattices if the site status are fed. Fortunately, the GCN
0.1. The lines are the real data, which is the average of captures the information about the connections between
20 bins. The colorful bands also mark the error bars. the sites, as well as global features like lattice percolation.
The location of the peak of the “W” shaped curve is the We show a possibility that the GCN can act as a general
percolation threshold, which is around pc = 0.593. In framework that can simultaneously detect the percola-
the simulation, to get a better result for the confusion tion threshold on different lattices in a supervised man-
method, in addition to putting pi in the embedding, we ner. The intersection in the “X” shaped output curves
 5

is the critical point. For unsupervised machine learning, [11] C. M. Fortuin and P. W. Kasteleyn, On the random-
the position of the “W”-shaped performance also reflects cluster model: I. Introduction and relation to other mod-
the predicted percolation threshold well. At the technical els, Physica 57, 536 (1972).
level, we build a graph neural network suitable for lattice [12] C. Fortuin, On the random-cluster model ii. the percola-
tion model, Physica 58, 393 (1972).
statistical physical models such as the percolation model. [13] M. E. Fisher, Magnetic critical point exponents—their in-
Our work is helpful for the extension of GCN to many terrelations and meaning, J. Appl. Phys. 38, 981 (1967).
percolation-related topics. Further, we build a more gen- [14] A. Coniglio and W. Klein, Clusters and Ising critical
eral neural network to train physical systems with fixed droplets: a renormalisation group approach, J. Phys. A
nodes but different topologies. Math. Theor. 13, 2775 (1980).
[15] C.-K. Hu, Percolation, clusters, and phase transitions in
Acknowledgments We thank for the valuable discus- spin models, Phys. Rev. B 29, 5103 (1984).
sion from Tzu-Chieh Wei about the confusion method, [16] J. Carrasquilla and R. G. Melko, Machine learning phases
and also thank Junyin Zhang, Bo Zhang and Longxiang of matter, Nature Physics 13, 431 (2017).
Liu for useful discussions. W.Z. was supported by the [17] G. Carleo and M. Troyer, Solving the Quantum Many-
Body Problem with Artificial Neural Networks, Sci-
Hefei National Research Center for Physical Sciences at
ence 355, 602 (2017), arXiv:1606.02318 [cond-mat,
the Microscale (KF2021002), and Peng Huanwu Center’s physics:quant-ph].
Visiting Scientist Program for 2022 by Institute of The- [18] G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld,
oretical Physics, Chinese Academy of Science, Beijing. N. Tishby, L. Vogt-Maranto, and L. Zdeborová, Machine
L.Z. and H.T. were supported by NSFC under Grant No. learning and the physical sciences, Rev. Mod. Phys. 91,
51901152. Y.D. is supported by the National Natural 045002 (2019).
Science Foundation of China under Grant No. 11625522, [19] L. Wang, Discovering phase transitions with unsuper-
vised learning, Phys. Rev. B 94, 195105 (2016).
the Science and Technology Committee of Shanghai un-
[20] L. van der Maaten and G. Hinton, Visualizing data us-
der grant No. 20DZ2210100, and the National Key R and ing t-sne, Journal of Machine Learning Research 9, 2579
D Program of China under Grant No. 2018YFA0306501. (2008).
[21] J. F. Rodriguez-Nieva and M. S. Scheurer, Identifying
topological order through unsupervised machine learn-
ing, Nat. Phys. 15, 790 (2019).
[22] M. S. Scheurer and R.-J. Slager, Unsupervised machine
∗
Corerespoinding author: [email protected] learning and band topology, Phys. Rev. Lett. 124, 226401
†
Corresponding author: [email protected] (2020).
[1] S. Broadbent and J. M. Hammersley, Percolation pro- [23] E. P. L. Van Nieuwenburg, Y.-H. Liu, and S. D. Huber,
cesses. i. crystals and mazes (1957). Learning phase transitions by confusion, Nat. Phys. 13,
[2] R. G. Larson, L. E. Scriven, and H. T. Davis, Percolation 435 (2017).
theory of residual phases in porous media, Nature 268, [24] J. Shen, W. Li, S. Deng, and T. Zhang, Supervised and
409 (1977). unsupervised learning of directed percolation, Phys. Rev.
[3] K. P. Krishnaraj and P. R. Nott, Coherent force chains E 103, 052140 (2021).
in disordered granular materials emerge from a percola- [25] W. Zhang, J. Liu, and T.-C. Wei, Machine learning of
tion of quasilinear clusters, Phys. Rev. Lett. 124, 198002 phase transitions in the percolation and X Y models,
(2020). Physical Review E 99, 032142 (2019).
[4] O. Riordan and L. Warnke, Explosive Percolation Is Con- [26] D. Bayo, A. Honecker, and R. A. Römer, Machine learn-
tinuous, Science 333, 322 (2011). ing the 2d percolation model, Journal of Physics: Con-
[5] M. Li, R.-R. Liu, L. Lü, M.-B. Hu, S. Xu, and Y.-C. ference Series 2207, 012057 (2022).
Zhang, Percolation on complex networks: Theory and [27] S. Cheng, F. He, H. Zhang, K.-D. Zhu, and Y. Shi,
application, Physics Reports 907, 1 (2021), percolation Machine learning percolation model, arXiv preprint
on complex networks: Theory and application. arXiv:2101.08928 (2021).
[6] M. E. J. Newman, Spread of epidemic disease on net- [28] W. Zhang, J. Liu, and T.-C. Wei, Machine learning of
works, Phys. Rev. E 66, 016128 (2002). phase transitions in the percolation and XY models,
[7] J. Fan, J. Meng, Y. Ashkenazy, S. Havlin, and Phys. Rev. E 99, 032142 (2019).
H. J. Schellnhuber, Climate network percolation re- [29] R. Xu, W. Fu, and H. Zhao, A new strategy in applying
veals the expansion and weakening of the tropical the learning machine to study phase transitions, arXiv
component under global warming, Proceedings of preprint arXiv:1901.00774 (2019).
the National Academy of Sciences 115, E12128 (2018), [30] J. Zhang, B. Zhang, J. Xu, W. Zhang, and Y. Deng,
https://www.pnas.org/doi/pdf/10.1073/pnas.1811068115. Machine learning for percolation utilizing auxiliary Ising
[8] M. Pant, D. Towsley, D. Englund, and S. Guha, Percola- variables, Physical Review E 105, 024144 (2022).
tion thresholds for photonic quantum computing, Nature [31] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu,
Communications 10, 1070 (2019). L. Wang, C. Li, and M. Sun, Graph neural networks:
[9] X. Feng, Y. Deng, and H. W. J. Blöte, Percolation transi- A review of methods and applications, AI Open 1, 57
tions in two dimensions, Phys. Rev. E 78, 031136 (2008). (2020).
[10] W. Huang, P. Hou, J. Wang, R. M. Ziff, and Y. Deng, [32] R. Ying, J. You, C. Morris, X. Ren, W. L. Hamilton, and
Critical percolation clusters in seven dimensions and on J. Leskovec, Hierarchical Graph Representation Learning
a complete graph, Phys. Rev. E 97, 022107 (2018). with Differentiable Pooling, arXiv:1806.08804 [cs, stat]
 6

(2019), arXiv:1806.08804 [cs, stat]. The matrix A and S are shown as follows,
[33] T. Xie and J. C. Grossman, Crystal graph convolutional
neural networks for an accurate and interpretable predic-
tion of material properties, Phys. Rev. Lett. 120, 145301
(2018).
[34] T. N. Kipf and M. Welling, Semi-Supervised Clas-
sification with Graph Convolutional Networks,
arXiv:1609.02907 [cs, stat] (2017), arXiv:1609.02907 [cs,
stat].
[35] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and
G. E. Dahl, Neural Message Passing for Quantum Chem-
istry, arXiv:1704.01212 [cs] (2017), arXiv:1704.01212    
0 1 1 1 0 0 1 0 0 10 0 0
[cs]. 1
 0 1 0 1 0 0 1 0
1
 0 0 0
1 1 0 0 0 1 0 0 1 00 1 0
   
1 0 0 0 1 1 1 0 0 00 0 1
I. ALGORITHMS
A(0) (0)
  
0
= 1 0 1 0 1 0 1 0 S = 10, 0 0

0 0 1 1 1 0 0 0 1 00 1 0
   
1 0 0 1 0 0 0 1 1 00 0 1
Algorithm 1 A batch of training 
0
 
1 0 0 1 0 1 0 1  00 0 1
Initialize conv num=L;pool num=k;len G=|V |
0 0 1 0 0 1 1 1 0 01 0 0
Input: percolation snapshots
Output: Category probability p1 , p2 (10)
1: data input
where Sij = 1 if and only if node i belongs to the cluster
Generate percolation graph representation G j or the super node j. Using the transformation S T AS,
∀i, j, Initialize {vi |vi ∈ V (G)}, {eij |eij ∈ E(G)} one gets a new adjacency matrix A(1) defined as
∀i, j, use FCN to get new node embedding h0vi
2: data processing
for num=0 to conv num do:
∀i, j, zvl i = concat(hlvi , hlvj , evi vj )
hlvi = hl−1 σ(M LP (zvl i ))  g(M LP (zvl i ))
P
vi +
for len G to 1 do:
Gk = Dif f P ool(Gk−1 )
3: data output
p1 , p2 = Sof tmax(M LP (Gk ))
end  
4 3 5 0
3 2 1 2
II. AN EXAMPLE OF HARD CLUSTER A(1) = S (0)T A(0) S (0) = , (11)
NUMBER ASSIGNMENT 5 1 4 2
0 2 2 0

1 2 3 1 2

4 5 6 1

3 4
7 8 9

(a) (b) (c)

FIG. 5. Graph collapse example. A 9-node graph is collapsed where 4, 3, 5 in the first row means the super node 1 is
into a 4-super-node graph. The three nodes (1, 2, 5) in blue connected to the super nodes 1, 2, 3, which is consistent
become a super node. The nodes in the pink shadow, green with the description in Fig. 5. A(1) is an new adjacency
shadow and yellow shadow are performed similarly matrix with weights.