Fast and Accurate Graph Stream Summarization

In the era of big data, data streams propose some technique challenges for existing systems. Furthermore, the traditional data stream is modeled as a sequence of isolated items, and the connections between the items are rarely considered. However, in many data stream applications, the connections often play important roles in data analysis, such as finding malicious attacks in the network traffic data, mining news spreading paths among the social network. In these cases the data is organized as graph streams. A graph stream is an unbounded sequence of items, in which each item is a vector with at least three fields (denoted by $(\langle s,d\rangle,w)$), where $\langle s,d\rangle$ represents an edge between nodes $s$ and $d$, and $w$ is the edge weight. These data items together form a dynamic graph that changes continuously and we call it streaming graph for convenience. Below we discuss three examples to demonstrate the usefulness of streaming graph problems.

These streaming graphs are very large and change fast. For example, in Twitter, there are about $100$ million user login data, with $500$ million tweets posted per day. For another example, in large ISP or data centers[1], there could be millions of packets every second in each link. The large volume and high dynamicity make it hard to store the graph streams efficiently with traditional data structures like adjacency lists or adjacency matrices. In the context of graph streams, there are two requirements for designing a new data structure : (1) the linear space cost (2) the constant update time. To meet these two requirements, we can either apply approximated query data structures for data streams, like the CM sketch[2], the CU sketch [3] and other sketches [4, 5], or use specialized graph summarization techniques such as gSketches [6], TCM [7] and gMatrix [8]. However, existing solutions either support limited query types or have poor query accuracy. For example, CM sketches and gSketches fail to answer queries involved with topology like reachability queries, successor queries and so on. Though TCM and gMatrix can support these queries, they have poor accuracy. More details about the related work are given in Section II. In this paper, we design a novel data structure–Graph Stream Sketch (GSS for short), which can support all kinds of queries over streaming graphs with controllable errors in query results. Both theoretical analysis and experiment results show that the accuracy of our method outperforms state-of-the-art by orders of magnitudes.

In this paper we propose GSS, which is an approximate query data structure for graph streams with linear memory usage, high update speed, high accuracy and supports all kinds of graph queries and algorithms like [9, 10, 11]. GSS can also be used in exiting distributed graph systems [12, 13, 14, 15]

Like TCM, GSS uses a hash function $H(\cdot)$ to compress the streaming graph $G$ into a smaller graph $G_{h}$ which is named a graph sketch. Each node $v$ in $G$ is mapped into a hash value $H(v)$. Nodes with the same hash value are combined into one node in $G_{h}$, and the edges connected to them are also aggregated. An example of the graph stream and the graph sketch can be referred in Fig.1 and Fig.2. The compression rate can be controlled by the size of the value range of $H(\cdot)$, which we represent with $M$. The higher the compression rate is, the lower the accuracy is, as more nodes and edges will be combined.

Different from TCM which uses an adjacency matrix to store the graph sketch $G_{h}$, GSS uses a novel data structure to store it. This data structure is specially designed for sparse graphs and can store a much bigger graph sketch with the same space. As the graph is sparse, the number of nodes is large, but each node is connected to few edges. Therefore, different from adjacency matrix which stores edges with the same source node / destination node in one row / column, we store edges with different source nodes / destination nodes in the one row / column, and distinguish them with fingerprints. Each edge in the graph sketch is mapped to a bucket in the matrix depending on its endpoints, and marked with a fingerprint pair. If the bucket it is mapped is already occupied by other edges, we store this edge in a buffer $B$, which is composed of adjacency lists. With a $m\times m$ matrix we can represent a graph sketch with at most $m\times F$ nodes in GSS, where $F$ is the size of the value range of the fingerprint (for example, a 16-bit fingerprint has $F=65536$). On the other hand, the adjacency matrix can only store a a graph sketch with at most $m$ nodes. With a much larger graph sketch, the accuracy is also much higher compared to TCM.

In GSS, the memory cost and update speed are greatly influenced by the size of the buffer $B$. As the buffer takes additional memory, and update speed in an adjacency list is linear with its size. In order to restrict its size, we propose a technique called square hashing. In this technique each edge is mapped to multiple buckets, and stored in the first empty one among them. This enlarges the chance that an edge finds an empty bucket. Besides, a few nodes in a sparse graph may still have very high degrees. If one node emits a lot of edges, these edges have high probability to evict each other when stored in one row. To solve this problem, In square hashing edges with source node $v$ are no longer mapped to one row, but $r$ rows, sharing memory with other source nodes. The higher degree a node has, the more buckets it may take. It is similar in the view of columns and destination nodes. This helps to ease the congestion brought by the skewness in node degrees. Experiments show that after this modification the buffer only stores less than $0.01\%$ of the edges in the graph stream.

The key contributions of this paper are as follows:

1. 1.

   We propose GSS, a novel data structure for graph stream summarization. It has small memory usage, high update speed, and supports almost all kinds of queries for graphs. Most important of all, it uses a combination of fingerprints and hash addresses to achieve very high accuracy.

2. 2.

   We propose a technique called square hashing in the implementation of GSS. It helps to decrease the buffer size, improve update speed and reduce memory cost. It also eases the influence brought by the skewness in node degrees.

3. 3.

   We define $3$ graph query primitives and give details about how GSS supports them. Almost all algorithms for graphs can be implemented with these primitives.

4. 4.

   We carry out theoretical analysis and extensive experiments to evaluate the performance of GSS, which show that when using 1/256 memory size of the state-of-the-art graph summarization algorithm, our algorithm still significantly outperforms it for most queries.

In the basic version, an edge is pushed into buffer $B$ if and only if its mapped position in the matrix $X$ has been occupied. The most intuitive solution is to find another bucket for it. Then where to find an empty bucket? We further notice the skewness in node degrees. In the real-world graphs, node degrees usually follow the power law distribution. In other words, a few nodes have very high degrees, while most nodes have small degrees. Consider a node $v$ that has $A$ out-going edges in the graph sketch $G_{h}$. For a $m\times m$ adjacency matrix $X$ in GSS (see Definition 5), there are at least $A-m$ edges that should be inserted into buffer $B$, as these $A$ edges must be mapped to the same row (in $X$) due to the same source vertex $v$. These high degree nodes lead to crowed rows and result in most left-over edges in buffer $B$. On the other hand, many other rows are uncrowded. We have the same observation for columns of matrix $X$. Is it possible to make use of the unoccupied positions in uncrowded rows/columns? It is the motivation of our first technique, called square hashing.

For each node with ID $H(v)=\langle h(v),f(v)\rangle$ in $G_{h}$, we compute a sequence of hash addresses $\{h_{i}(v)|1\leqslant i\leqslant r\}(0\leqslant h_{i}(v)<m)$ for it. Edge $\overrightarrow{H(s),H(d)}$ is stored in the first empty bucket among the $r\times r$ buckets with addresses

$\{(h_{i}(s),h_{j}(d))|(1\leqslant i\leqslant r,1\leqslant j\leqslant r)\}$

where $h_{i}(s)$ is the row index and $h_{j}(d)$ is the column index. We call these buckets mapped buckets for convenience. Note that we consider row-first layout when selecting the first empty bucket.

The following issue is how to generate a good hash address sequence $\{h_{i}(v)|1\leq i\leq r\}$ for a vertex $v$. There are two requirements:

Independent: For two nodes $v_{1}$ and $v_{2}$, we use $P$ to represent the probability that $\forall 1\leq i\leq r,h_{i}(v_{1})=h_{i}(v_{2})$. Then we have $P=\prod_{i=1}^{r}Pr(h_{i}(v_{1})=h_{i}(v_{2}))$. In other words, the randomness of each address in the sequence will not be influenced by others. This requirement will help to maximize the chance that an edge finds an empty bucket among the $r\times r$ mapped buckets.

Reversible: Given a bucket in row $R$ and column $C$ and the content in it, we are able to recover the representation of the edge $e$ in the graph sketch $G_{h}$: $\overrightarrow{H(s),H(d)}$, where $e$ is the edge in that bucket. This property of indexing is needed in the 1-hop successor query and the 1-hop precursor query. As in these queries, we need to check the potential buckets to see if they contain edges connected to the queried node $v$ and retrieve the other end point in each qualified bucket.

To meet the above requirements, we propose to use linear congruence method[22] to generate a sequence of $r$ random values $\left\{q_{i}(v)|1\leqslant i\leqslant r\right\}$ with $f(v)$ as seeds. We call this sequence the linear congruential (LR) sequence for convenience. The linear congruence method is as following: select a timer $a$, small prime $b$ and a module $p$, then

By choosing $a$, $b$ and $p$ carefully, we can make sure the cycle of the sequence we generate is much larger than $r$, and there will be no repetitive numbers in the sequence [22]. Then we generate a sequence of hash addresses as following:

When storing edge $\overrightarrow{H(s),H(d)}$ in the matrix, besides storing the pair of fingerprints and the edge weight, we also store an index pair $(i_{s},i_{d})$, supposing that the bucket that contains this room has an address $(h_{i_{s}}(s),h_{i_{d}}(d))$. As the length of the sequence, $r$, is small, the length of each index will be less than $4$ bits. Therefore storing such a pair will cost little.

Note that the hash sequence $\left\{q_{i}(v)|1\leqslant i\leqslant r\right\}$ generated by the linear congruence method are both independent and reversible. The independence property has been proved in [8]. We show how to recover the original hash value $H(v)$ based on the $f(v)$, $h_{i}(v)$ and the index $i$ as follows. First, we compute the LR sequence $\{q_{i}(v)\}$ with $f(v)$ following equation 4. Second we use the equation $(h(v)+q_{i}(v))\%m=h_{i}(v)$ to compute the original hash address $h(v)$. As $h(v)<m$, the equation has unique solution. At last we use $H(v)=h(v)\times F+f(v)$ to compute $H(v)$. Given a bucket in the matrix, the fingerprint pair $(f(s),f(d))$ and the index pair $(i_{s},i_{d})$ are all stored in it, and we have $h_{i_{s}}(s)=R$, $h_{i_{d}}(d)=C$, where $R$ and $C$ are the row index and the column index of the bucket in the matrix, respectively. Therefore we can retrieve both $H(s)$ and $H(d)$ as above.

Compared to the basic version, in the modified version all edges are stored in the matrix, and the number of memory accesses we need to find an edge in the matrix is within $2^{2}=4$. In fact in the example we only need one memory access to find most edges, and $2$ for a few ones.

In the following, we illustrate the four basic operators in this data structure GSS.

Edge Updating: When a new item $(s,d,t;w)$ comes in the graph stream $S$, we map it to edge $\overrightarrow{H(s),H(d)}$ in the graph sketch $G_{h}$ with weight $w$. Then we compute two hash address sequences $\{h_{i}(s)\}$ and $\{h_{i}(d)\}$ and check the $r^{2}$ mapped buckets with addresses $\{(h_{i}(s),h_{j}(d))|1\leqslant i\leqslant r,1\leqslant j\leqslant r\}$ one by one. For a bucket in row $h_{i_{s}}(s)$ and column $h_{i_{d}}(d)$, if it is empty, we store the fingerprint pair $(f(s),f(d))$ and the index pair $(i_{s},i_{d})$ and weights $w$ in it, and end the procedure. If it is not empty, we check the fingerprint pair $(f(s^{\prime}),f(d^{\prime}))$ and the index pair $(i_{s}^{\prime},i_{d}^{\prime})$ stored in the bucket. If the fingerprint pair and the index pair are all equal to the corresponding pairs of the new inserted edge $\overrightarrow{H(s),H(d)}$, we add $w$ to the weights in it, and end the procedure. Otherwise it means this bucket has been occupied by other edges and we consider other hash addresses following the hash sequence. If all $r^{2}$ buckets have been occupied, we store edge $\overrightarrow{H(s),H(d)}$ with weight $w$ in the buffer $B$, like the basic version of GSS.

Graph Query Primitives: The three graph query primitives are supported in the modified data structure as follows:

Edge Query: When querying an edge $e=\overrightarrow{s,d}$, we map it to edge $\overrightarrow{H(s),H(d)}$ in the graph sketch, and use the same square hashingmethod to find the $r^{2}$ mapped buckets and check them one by one. Once we find a bucket in row $h_{i_{s}}(s)$ and column $h_{i_{d}}(d)$ which contains the fingerprint pair $(f(s),f(d))$ and the index pair $(i_{s},i_{d})$, we return its weight as the result. If we find no results in the $r^{2}$ buckets, we search the buffer for edge $\overrightarrow{H(s),H(d)}$ and return its weights. If we still can not find it, we return $-1$.

1-hop Successor Query: to find the 1-hop successors of node $v$, we map it to node $H(v)$ in $G_{h}$. Then we compute its hash address sequence ${h_{i}(v)}$ according to $H(v)$, and check the $r$ rows with index $h_{i}(v),(1\leqslant i\leqslant r)$. If a bucket in row $h_{i_{s}}(v)$, column $C$ contains fingerprint pair $((f(v),f(x))$ and index pair $(i_{s},i_{d})$ where $f(x)$ is any integer in range $[0,F)$ and $i_{d}$ is any integer in range $[1,r]$, we use $f(x)$, $i_{d}$ and $C$ to compute $H(x)$ as stated above. Then we add $H(x)$ to the 1-hop successor set $SS$. After searching the $r$ rows, we also need to check the buffer to see if there are any edges with source node $H(v)$ and add their destination node to $SS$. We return $-1$ if we find no result, otherwise we obtain the original node IDs from $SS$ by accessing the hash table $\langle H(v),v\rangle$.

1-hop Precursor Query: to answer an 1-hop precursor query, we have the analogue operations with $1$-hop Successor Query if we switch the columns and the rows in the matrix $X$. The details are omitted due to space limit.

After applying square hashing, the edges with source node $H(v)$ in $G_{h}$ are on longer stored in a single row, but spread over $r$ rows with addresses $h_{i}(v)(1\leqslant i\leqslant r)$. Similarly, edges with destination node $H(v)$ are stored in the $r$ different columns. These rows or columns are shared by the edges with different source nodes or destination nodes. The higher degree a node has, the more buckets its edges may take. This eases the congestion brought by skewness in node degrees. Moreover, as each bucket has multiple mapped buckets, it has higher probability to find an empty one. Obviously, square hashing will reduce the number of left-over edges.

There are some other improvements which can be implemented to GSS.

As stated above, GSS has $O(|E|)$ memory cost and constant update speed. The memory cost of GSS is $O(|E_{h}|+|B|)$, to be precise, where $|E_{h}|$ is the number of edges in the graph sketch $G_{h}$ and $|B|$ is the size of buffer. When we use hash table to store the original ID, additional $O(|V|)$ memory is needed, but the overall memory cost is still $O(|E|)$. The update time cost is $O(k+\frac{|B|}{|E_{h}|}|B|)$, where $k$ is the number of sampled buckets and is a small constant. When an edge is stored in the matrix, we only need to check at most $k$ candidate buckets, which takes $O(k)$ time. Each edge has probability $\frac{|B|}{|E_{h}|}$ to be stored in the buffer. When it is stored in the buffer, the update takes additional $O(|B|)$ time, as the buffer is an adjacency list. In implementations the buffer stores $0$ edges in most cases, which will be shown in section VI-D and VII-G. Therefore $\frac{|B|}{|E_{h}|}|B|$ is also a small constant. When it is necessary to store the ID of nodes in applications, one insertion to the hash table is needed, which also takes constant time. Overall, the update time cost is $O(1)$.

The time cost of queries is based on the algorithms we use. We consider the time cost of the operators as an evaluation. The time cost of the edge query operator is the same as the update, and the time cost of the 1-hop successor query and 1-hop precursor query is $O(rm+|B|)$, where $m$ is the side length of the matrix and $r$ is the length of the hash address sequence.

In this part we evaluate the accuracy of GSS. Before we analyze the probability of errors, we first propose the following theorem:

As the buffer is an adjacency list that stores edges in $G_{h}$ accurately, we only need to check the matrix. If we want to prove the storage of the graph sketch is accurate, we need to prove that if such collision happens to $e_{1}$ and $e_{2}$, we have $e_{1}=e_{2}$ in $G_{h}$, in other words, $H(s_{1})=H(s_{2}),H(d_{1})=H(d_{2})$ .We assume that the bucket contains the wights of $e_{1}$ and $e_{2}$ is in row $R$ and column $C$ in the matrix. Obviously $e_{1}$ and $e_{2}$ must have the same fingerprint pair, otherwise it will be easy for us to differentiate them. With the same fingerprints, these two edges will produce the same LR sequences $\{q(s)\}$ and $\{q(d)\}$. Moreover, this bucket must have the same index pair for these two edges, and we represent this pair with $(c_{1},c_{2})$. Then we have

and

With these equations, we can get that $h(d_{1})=h(d_{2})$, $h(s_{1})=h(s_{2})$ when $0\leqslant h(s_{1}),h(s_{2}),h(d_{1}),h(d_{2})<m$. With the same fingerprint pair and hash values, we have $H(s_{1})=H(s_{2})$, $H(d_{1})=H(d_{2})$. $e_{1}$ and $e_{2}$ are the same edge in $G_{h}$. Therefore the storage of $G_{h}$ is accurate.

This theorem means we only need to consider the procedure of mapping $G$ to $G_{h}$, as all errors happen in this procedure. We use $\hat{P}$ to represent the probability of the following event:

We set $P=1-\hat{P}$, and $P$ is the main component of the error rate of all the $3$ graph query primitives.

In the edge query, $P$ is just the correct rate. In the 1-hop successor query for a node $v$, the correct rate is ${P}^{|V|-d}$, where $|V|$ is the number of nodes in $G$, and $d$ is the out-degree of the queried node. Because we will get a correct answer if and only if for each $v^{\prime}$ in $G$ which is not a 1-hop successor of $v$, $(v,v^{\prime})$ does not collide with any existing edges, and the probability of such an event is $P$. It is the same in the 1-hop precursor query. Therefore we need to compute $P$ to evaluate the accuracy of GSS.

Now we show the probability that an edge $e$ suffers from edge collision, $\hat{P}$. For $e=\overrightarrow{s,d}$ in $G$, we assume there are $D$ edges with source node $s$ or destination node $d$ in $G$ besides $e$, and there are totally $|E|$ edges in $G$. We represent the size of the value range of the map function $H(\cdot)$ with $M$.

For an edge share no common endpoints with $e$ , it will collide with $e$ when both its source node and destination node collide with the corresponding node of $e$. The probability that it collides with $e$ in map function $H(\cdot)$ is:

The probability that all the $|E|-D$ edges have no collisions with $e$ is

For those $D$ edges connected to $e$, as one of the two end points is the same, the probability that such an edge has a collision with $e$ is

The probability that all the $D$ edges have no collisions with $e$ is

Therefore the correct rate of $e$, in other words, all the $|E|$ edges do not have collisions with $e$ in mapping is

And $\hat{P}=1-P$. In GSS we have $M=m\times F$, where $m$ is the length of the matrix, and $F$ is the maximum size of the fingerprints. The above correct rate is usually very high in applications. For example, suppose that the fingerprints we use are $8$-bit, in other words, $F=256$, and when querying an edge $e$, we have $|E|=5\times 10^{5}$,$D=200$. We use a matrix with side length $m=1000$. Then the correct rate of this edge query is $e^{-0.00078}=0.9992$. On the other hand, in TCM the accuracy analysis is the same as GSS  but we have $M=m$. This lead to the difference on accuracy with the same size of matrix. With the same matrix size, TCM only has a probability of $0.497$ to get a correct weight for $e$.

After all the improvements, the buffer in GSSis very small. The mathematical expression of the buffer size is very complicated and is influenced by many details of the graph. Therefore we give an expression of the probability that a new edge $e$ becomes a left-over edge, which means inserted into the buffer, as a measurement. Assuming that there are already $N$ different edges in the graph stream, and among them $D$ edges have common source node or common destination node with $e$. The length of the matrix is $m$, and each bucket in the matrix has $l$ rooms. For each node we compute a hash address sequence with length $r$. For each edge we choose $k$ candidate buckets among the $r^{2}$ mapped buckets. Then the probability that $e$ becomes a left-over edge is: For each candidate bucket of $e$, as the $N-D$ non-adjacent edges are randomly inserted into the matrix with area $m^{2}$, the probability that there are $a_{1}$ non-adjacent edges inserted into it is:

As the $D$ adjacent edges are randomly inserted in an area of $r\times m$ ($r$ rows or $r$ columns in the matrix), the probability that there are $a_{2}$ adjacent edges inserted into this bucket is:

The probability that there are already $n$ edges inserted into this bucket is:

The probability that there are less than $l$ edges inserted into this bucket is:

This is also the lower bound that the bucket is still available for $e$. The probability that $e$ can not be inserted into the matrix is the probability that all the $k$ candidate buckets are not available, which is:

where

Notice that this is an upper bound as we ignore collisions in the map procedure from $G$ to $G_{h}$. This probability is rather small. For example if $N=1\times 10^{6}$, $D=10^{4}$, we still set the side length of the matrix to $w=1000$, and set $r=8$, $l=3$, $k=8$, the upper bound probability of insertion failure is only $0.002$. Experiments show that when the size of matrix is nearly equal to the number of edges, there will be almost no edges inserted into the buffer.