Efficient Representation of Interaction Patterns with Hyperbolic Hierarchical Clustering for Classification of Users on Twitter

Earlier works that studied spam on Twitter (Mccord and Chuah 2011) leveraged user characteristics such as number of followers, and tweet content to generate features. These features were used to train a random forest classifier to detect spamming accounts. Follow-up works (Lee and Kim 2014) aimed to identify malicious accounts created in a short period of time by using account names. They compare algorithm-made names with man-made names by clustering accounts sharing similar name-based features. The work by Wei et al. (Wei et al. 2016) uses temporal sentiment analysis to differentiate suspended users from non-suspended users with the help of statistical techniques like Naive Bayes classifier and SVM.

With claims that Twitter had been influencing voter sentiment during the U.S. elections, there have been focused attempts at characterizing Twitter’s part in democratic processes in many nations (Strandberg 2013; Knight 2012; Dzisah 2018). Notable studies on characterizing users based on Twitter’s moderation decisions (Le et al. 2019) show that the malicious communities suspended by Twitter exhibit a considerable difference from regular accounts. This class of works combines Twitter and elections to understand different user groups on the platform, spread of sentiment and news on the network and echo chambers and their effects, to minimize the effect Twitter has on democratic processes. In other works (Chowdhury et al. 2021), authors group Twitter users into communities based on their retweet and mention networks and analyze different characteristics such as popular tweeters, domains, and hashtags. They found that malicious and regular accounts participate in communities which exhibit significant differences in terms of popular account and hashtag usage.

Work has also been done on identifying and characterizing deleted users on social media platforms. Authors in (Bastos 2021) found that a significant proportion of accounts involved in political discourse on Twitter revolving around the Brexit referendum campaign were deleted from the platform. Volkova et al. used profile, network and behavior clues, sentiment and emotion features, text embeddings and topics to detect accounts deleted from Twitter which were active in the context of the Russian-Ukranian crisis (Volkova and Bell 2016). Twitter itself removed over 2 million accounts from the platform which were suspected to be fake. These accounts were allegedly giving misleading follower counts for some accounts (Dave 2018).

Hyperbolic Hierarchical Clustering (HypHC) (Chami et al. 2020) is a key component of our work. We use this technique to reduce the dimensionality of our user representations for efficient computation. The following sub-section introduces previous research work on hyperbolic manifolds and their applications.

In the work (Nickel and Kiela 2017) authors claim that the ability of embeddings to model complex patterns is bounded by the dimensionality of the embedding space, because of which it is impossible to extract embeddings of large graph-structured data without any loss of information. So, to increase the representation capacity of embedding methods, they used hyperbolic spaces. Representations in hyperbolic spaces are capable of effectively capturing the underlying hierarchical structures in data at lower dimensions (Feng et al. 2020).

Feng et al. (Feng et al. 2020) leveraged this property of hyperbolic spaces to build a hyperbolic metric emebedding model, which projects location-based check-in data of users from social media into the hyperbolic space to predict the next POI (point of interest) for a user. Wang et al. (Wang et al. 2020b) proposed a hyperbolic geometry representation learning model to link user identities across different social network platforms.

Hyperbolic manifolds have also been used to embed knowledge graphs (Wang et al. 2020a), images (Khrulkov et al. 2020) and words (Tifrea et al. 2018) in far lower dimensions. Question-answering (Tay et al. 2018) and clustering (Monath et al. 2019) models have also leveraged hyperbolic manifolds to improve their performance.

During the 2019 General Elections in India, contesting parties have been known to maintain an IT cell (Chattopadhyay 2020; Campbell-Smith and Bradshaw 2019). These IT cells, along with a dedicated team of supporters work to push the agenda of their parties as much as possible. As mentioned in (Campbell-Smith and Bradshaw 2019), these IT cells are known to propagate particular agendas on OSNs by engaging with posts by a certain account. So, artificial engagement in the context of the General Elections revolves around boosting the popularity and pushing the ideologies of a particular leader or party.

To identify the top leaders, we curate a list of users whose content is widely shared across the platform. We call them Influencers in our work because of the effect these users have on the content shared on the platorm. We particularly look at the engagement on the platform as a result of their tweets. Influencers need not be political leaders, but in the context of the 2019 General Elections, we observed that most Influencers are political leaders or political party handles. We use these Influencers to generate our interaction features. Table 1 summarizes the notations used henceforth.

Retweets are a great way to quickly engage and amplify the reach of a tweet. Those retweets without any text of their own are just endorsements (Designer 2021; Cork and Eddy 2017; Kim and Yoo 2012). We take all retweets from the data and curate a list of user profiles $I^{\prime}$ whose tweets have been retweeted. For each such user in $I^{\prime}$, we keep track of how many users in the dataset retweeted their tweets (i.e. $rt\_score$). We then consider the top $p$ user profiles having the highest number of retweets as Influencers to form the set $I$.

The set of users interacting with Influencers in $I$ is given by $U=\{u_{1},u_{2},\dots\}$ where each user in $U$ has engaged with at least one Influencer in $I$ by retweeting at least one of their tweets. For each Influencer, we define $T_{y}=\{t_{y1},t_{y2}\dots\}$ to be the set of all tweets by Influencer $i_{y}$. For each pair $(u_{x},i_{y})$ where user $u_{x}\in U$ and Influencer $i_{y}\in I$, we look at those tweets by user $u_{x}$ which are retweets of any tweet by Influencer $i_{y}$, i.e. retweet of any tweet in $T_{y}$. We thus have a set of retweets $R_{xy}=\{$set of all $r_{xt_{yk}}\}$ where $r_{xt_{yk}}$ is a retweet of Influencer $i_{y}$’s tweet $t_{yk}\in T_{y}$ by user $u_{x}$. If a user $u_{x}$ has never retweeted a tweet by Influencer $i_{y}$, then $R_{xy}=\{\}$. In order to quantify a particular user’s ($u_{x}$) interaction with any Influencer ($i_{y}$), we use two values:

1. (1)

   The first value we use is the delay, or time lag in retweeting. For each retweet $r_{xt_{yk}}$ in $R_{xy}$, we define the corresponding delay $\delta_{xt_{yk}}$ to be the absolute value of difference in seconds between time of the original tweet ($t_{yk}$) and the time of retweet of that tweet ($r_{xt_{yk}}$), i.e. $\delta_{xt_{yk}}=$ (time of retweet $r_{xt_{yk}}$ - time of original tweet $t_{yk}$). Figure 1 shows an illustrative demonstration of how each such $\delta$ is calculated. Thus, for each pair $(u_{x},i_{y})$ we have a set $D_{xy}$ which is the set of all $\delta_{xt_{yk}}$. $D_{xy}$ forms the set of delays for each retweet by user $u_{x}$ of Influencer $i_{y}$. We take the median of the delays in $D_{xy}$ to get the final delay $\Delta_{xy}$.

2. (2)

   The second value that we use is the number of times the user has retweeted that Influencer’s tweets. We define $n_{xy}$ to be the number of elements in $R_{xy}$, i.e. the number of times user $u_{x}$ retweeted a tweet by Influencer $i_{y}$.

Thus for each pair $(u_{x},i_{y})$, we have a two-element vector $\vec{v}_{xy}$ which has two values: delay $\Delta_{xy}$ and the number of retweets $n_{xy}$. We form the interaction feature vector $\vec{V}_{x}$ for each user $u_{x}$ by concatenating all such vectors $\vec{v}_{xy}$ obtained for each corresponding Influencer $i_{y}$ where $i_{y}\in I$. In case a user has never interacted with an Influencer (i.e. $R_{xy}=\{\}$), we set $\Delta_{xy}$ to a large negative value, and $n_{xy}$ to 0. We choose a large negative $\Delta_{xy}$ in this case because such a value would never appear if $R_{xy}$ was not empty, thus achieving a good separation in the feature space. Algorithm 1 describes the above explained feature engineering process in a pseudo-code format.

With the interaction feature vector $\vec{V}_{x}$, we quantify each user’s interaction with the top handles. If there is a group of users colluding to interact with an Influencer account, their interaction feature vectors would look similar. Thus our features will help to capture such groups of collusive accounts.

In order to generate interaction features we have to choose the number of Influencers $p$ to include in the set $I$. We observed the trends of number of unique users added to the set $U$ for each new Influencer added to the set $I$. We wish to choose $p$ high enough that a large number of users are incorporated. A larger set of users means different subgroups of these users interact with the tweets of different Influencers, thus capturing a larger number of collusive groups. However, at the same time we do not wish to have a large $p$ because we only want the top tweeting Influencers, i.e. the most popular ones, who have a relatively large number of users retweeting their tweets. In order to achieve this balance, we study the relationship between the number of Influencers and number of users as depicted in Figure 2. In Figure 2(a), each point on the x-axis represents a particular Influencer, and the y-axis depicts the number of unique users retweeting that particular Influencer (i.e. $rt\_score$). This indicates how many unique users are interacting with each Influencer. Higher the value, more popular the Influencer.

To get a sense of the diversity of users incorporated by choosing a particular $p$, we plot Figure 2(b). In Figure 2(b), the x-axis represents the number of Influencers $p$, and the y-axis represents the size of $U$ (or $|U|$) for a particular value of $p$. It is important to note that Figure 2(b) is not merely a cumulative plot of Figure 2(a) because there will be users who retweet the tweets of multiple Influencers. This graph is increasing, because as we add more Influencers to the set $I$, we incorporate more users in the set $U$. The amount of increase in the y-value of the graph as the x-value changes from ${p}$ to ${p+1}$ is the number of users added in $U$ when we increase the number of Influencers by one. To better visualise the effect of increase in $p$ on $|U|$ we plot Figure 2(c).

Each value in Figure 2.c indicates the additional number of users that are included for each new Influencer added to I. We reiterate that there will be users who retweet the tweets of multiple Influencers, and thus Figure 2(c) is different from Figure 2(a). By increasing the value of our final chosen $p$, we include more spikes from Figure 2(c), which means we incorporate a more diverse set of users in $U$. In Figure 2(c), we observe that after around 300 Influencers, the additional number of users that are included for each new Influencer added reduces. We hypothesize this to be the optimal value of $p$. To verify the validity of this hypothesis, we chose values of $p$ at intervals of 50 between 100 and 600 and found best results on our classifiers (the same classifiers described in Section 6.1) with $p$=300. This confirmed our hypothesis. By choosing $p=300$ (as depicted with a vertical line in Figure 2), we end up with $U$ containing 59,260 users. The class distribution of our final $U$ is depicted in Table 2. Note that we refer to our proposed user interaction features as $F$ henceforth.

While comparing the different classes of users, we trained each model to classify the users in a one vs two fashion by training for suspended vs (regular + deleted), regular vs (deleted + suspended) and deleted vs (regular + suspended) separations. We also trained each model to classify the users in a one vs one (i.e. binary) fashion by training for suspended vs deleted, suspended vs regular and deleted vs regular separations.

We use two types of classifier models: deep learning based and tree based. For the deep learning based classifiers, we use a deep neural network and an LSTM. The results are presented after appropriate hyperparameter tuning in the loss function, activation function, the optimizer used, number of layers and the number of epochs. For the tree based classifiers, we use lightGBM (LGBM), XGBoost (XGB), Gradient Boosting Classifier (GBC) and the Random Forest Classifier (RFC). We use Grid Search to find the best set of hyperparameters for the tree based models. To account for class imbalance, we balance our training dataset using SMOTE (Chawla et al. 2002).

To further evaluate the performance of HypHC, we compare the distances between the centers of the three classes after performing dimensionality reduction of the interaction features $F$ using HypHC, SE and FA. We take each of these three representations, and standardize each feature by removing the mean and scaling to unit variance. Table 4 shows the cosine distances between the centers of each of the three classes in the 60 dimensional space that was generated by each of the dimensionality reduction methods. It is immediately obvious that HypHC is able to achieve the highest and most uniform separation between the classes. This ratifies our claim that HypHC is able to obtain the best separation between the classes.

Our observations from Sections 6.1 and 6.2 show the effectiveness of our interaction features, and the valuable advantage of using HypHC in our pipeline. On comparing the results of all the classifiers for the dimensionality reduced ($HypHC$) and high dimensional ($U+F$) features, we can see that the HypHC features often outperform the original features despite being at a much lower dimension. The added bonus of using lower dimensional data is decreased storage space and lower computation cost and time.