Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Models

Let $\mathcal{G}_{t}=(\mathcal{V},\mathcal{E})$ denote a traffic network at time step $t$, where $\mathcal{V}$ is a set of $n$ nodes (e.g., regions, road segments, roadway sensors, etc.) and $\mathcal{E}$ is a set of edges. The construction of $\mathcal{G}_{t}$ can be categorized into two types, (1) prior-based, which pre-define $\mathcal{G}_{t}$ based on metrics such as geographical proximity and similarity [10], and (2) learning-based, which automatically learns $\mathcal{G}_{t}$ in an end-to-end way [2]. Note the $\mathcal{G}_{t}$ can be static or time-evolving depending on the forecasting model. We denote $\mathbf{X}_{t}=(\mathbf{x}_{1,t},\mathbf{x}_{2,t},\cdots,\mathbf{x}_{n,t})$ as the spatiotemporal features associated to $\mathcal{G}_{t}$, where $\mathbf{x}_{i,t}\in\mathbb{R}^{c}$ represents the $c$-dimensional time-varying traffic conditions (e.g., traffic volume, traffic speed) and contextual features (e.g., weather, surrounding POIs) of node $v_{i}\in\mathcal{V}$ at $t$. The spatiotemporal traffic forecasting problem aims to predict traffic states for all $v_{i}\in\mathcal{V}$ over the next $\tau$ time steps,

where $\mathbfcal{H}_{t-T+1:t}=\{(\mathbf{X}_{t-T+1},\mathcal{G}_{t-T+1}),\dots,(\mathbf{X}_{t},\mathcal{G}_{t})\}$ denotes the traffic states contains input features and the traffic network in previous $T$ time steps, $f_{\theta}(\cdot)$ is the spatiotemporal traffic forecasting model parameterized by $\theta$, and $\mathbf{\hat{Y}}_{t+1:t+\tau}=\{\mathbf{\hat{Y}}_{t+1},\mathbf{\hat{Y}}_{t+2},\cdots,\mathbf{\hat{Y}}_{t+\tau}\}$ is the estimated traffic conditions of interest of $\mathcal{V}$ from time step $t+1$ to $t+\tau$. We denote $\mathbf{Y}_{t+1:t+\tau}=\{\mathbf{Y}_{t+1},\mathbf{Y}_{t+2},\cdots,\mathbf{Y}_{t+\tau}\}$ as the ground truth of $\mathbfcal{H}_{t-T+1:t}.$

Note the above formulation is consistent with the state-of-the-art Graph Neural Network (GNN) based spatiotemporal traffic forecasting models [2, 10, 11, 12], and is also generalizable to other variants such as Convolutional Neural Network (CNN) based approaches [13].

Given a machine learning model, adversarial attack aims to mislead the model to derive biased predictions by generating the optimal adversarial example

where $x^{\prime}$ is the adversarial example with maximum bound $\varepsilon$ under $L_{p}$ norm to guarantee the perturbation is imperceptible to human, and $y$ is the ground truth of clean example $x$.

Various gradient-based methods have been proposed to generate adversarial examples, such as FGSM [14], PGD [8], MIM [9], etc. For instance, the adversarial example $x^{\prime}=x+\varepsilon\text{sign}(\nabla_{x}\mathcal{L}_{CE}(x,y;\theta))$ in FGSM, where $\text{sign}(\cdot)$ is the Signum function and $\mathcal{L}_{CE}(\cdot)$ is the cross entropy loss.

Note the adversarial attack happened in the testing stage, and the attackers cannot manipulate the forecasting model or its output. On the benign testing set, the forecasting model can perform well. Based on the amount of information the attacker can access in the testing stage, the adversarial attack can be categorized into three classes. White-box attack. The attacker can fully access the target model, including the model architecture, the model parameters, gradients, model outputs, the input traffic states, and the corresponding labels. Grey-box attack. The attacker can partially access the system, including the target model and the input traffic states, but without the labels. Black-box attack. The attacker can only access the input traffic states, query the outputs of the target model or leverage a surrogate model to craft the adversarial examples.

This work aims to apply adversarial attacks to spatiotemporal traffic forecasting models. We first define the adversarial traffic state as follow,

where $S_{t}\in\{0,1\}^{n\times n}$ is a diagonal matrix with $i$th diagonal element indicating whether node $i$ is a victim node, and $\mathbf{X}^{\prime}_{t}$ is the perturbed spatiotemporal feature named adversarial spatiotemporal feature. We restrict the adversarial traffic state by the victim node budget $\eta$ and the perturbation budget $\varepsilon$.

Note that following the definition of adversarial attack, we leave the topology of $\mathcal{G}_{t}$ immutable as we regard the adjacency relationship as a part of the model parameter that may be automatically learned in an end-to-end way.

Attack goal. The attacker aims to craft adversarial traffic states to fool the spatiotemporal forecasting model to derive biased predictions. Formally, given a spatiotemporal forecasting model $f_{\theta}(\cdot)$, the adversarial attack against spatiotemporal traffic forecasting is defined as

where $\mathcal{T}_{test}$ and $\mathcal{T}_{train}$ denote the set of time steps of all testing and training samples, respectively. $\mathcal{L}(\cdot)$ is the loss function measuring the distance between the predicted traffic states and ground truth, and $\theta^{\ast}$ is optimal parameters learned during the training stage.

Since the ground truth (i.e., future traffic states) under the spatiotemporal traffic forecasting setting is unavailable at run-time, the practical adversarial spatiotemporal attack primarily falls into the grey-box attack setting.

However, investigating white-box attacks is still beneficial to help us understand how adversarial attack works and can help improve the robustness of spatiotemporal traffic forecasting models (e.g., apply adversarial training). We discuss how to extend our proposed adversarial attack framework to white-box and black-box settings in Section 3.2.

One unique characteristic that distinguishes attacking spatiotemporal forecasting from conventional classification tasks is the inaccessibility of ground truth at the test phase. Therefore, we first construct future traffic states’ surrogate label to guide the attack direction,

where $g_{\phi}(\cdot)$ is a generalized function (e.g., $\tanh(\cdot)$, $\sin{(\cdot)}$, $f_{\theta}(\cdot))$, $\mathcal{\delta}_{t+1:t+\tau}$ are random variables sampled from a probability distribution $\pi(\mathcal{\delta}_{t+1:t+\tau})$ to increase the diversity of the attack direction. In our implementation, we derive $\phi$ based on the pre-trained forecasting model parameter $\theta^{\ast}$, and $\mathcal{\delta}_{t+1:t+\tau}\sim U(-\varepsilon/10,\varepsilon/10)$. In the real-world production [5], the forecasting models are usually updated in an online fashion (e.g., per hours). Therefore, we estimate the missing latest traffic states based on previous input data, $\mathbfcal{\tilde{H}}_{t}=g_{\varphi}(\mathbfcal{H}_{t-1})$, where $g_{\varphi}(\cdot)$ is the estimation function parameterized by $\varphi$. For simplicity, we directly obtain $\varphi$ from the pre-trained traffic forecasting model $f_{\theta^{\ast}}(\cdot)$.

With the surrogate traffic state label $\mathbf{\tilde{Y}}_{t+1:t+\tau}$, we derive the time-dependent node saliency (TDNS) for each node as

where $\mathcal{L}(f_{\theta}(\mathbfcal{\tilde{H}}_{t-T+1:t}),\mathbf{\tilde{Y}}_{t+1:t+\tau})$ is the loss function and $\sigma$ is the activation function. Intuitively, $\mathcal{M}_{t}$ reveals the node-wise loss impact with the same degree of perturbations. Note depending on the time step $t$, $\mathcal{M}_{t}$ may vary. A similar idea also has been adopted to identify static pixel saliency for image classification [15].

More in detail, the loss function $\mathcal{L}(f_{\theta}(\mathbfcal{\tilde{H}}_{t-T+1:t}),\mathbf{\tilde{Y}}_{t+1:t+\tau})$ in Equation 6 is updated by the iterative gradient-based adversarial method [8],

where $\mathbfcal{H^{\prime}}_{t-T+1:t}^{(i)}$ is adversarial traffic states at $i$-th iteration, $\alpha$ is the step size, and $\text{clip}_{\mathbf{X^{\prime}}_{t-T+1},\varepsilon}(\cdot)$ is the project operation which clips the spatiotemporal feature with maximum perturbation bound $\varepsilon$. Note $\mathbfcal{H^{\prime}}_{t-T+1:t}^{(0)}=\mathbfcal{\tilde{H}}_{t-T+1:t}$.

For each batch of data $\left\{(\mathbfcal{\tilde{H}}_{t-T+1:t},\mathbf{\tilde{Y}}_{t+1:t+\tau})_{(j)}\right\}_{j=1}^{\gamma}$, the time-dependent node saliency gradient is derived by

where $\gamma$ is the batch size. We use the RELU activation function to compute the non-negative saliency score for each time step,

Finally, we obtain the set of victim node $S_{t}$ based on $\mathcal{M}_{t}$,

where $s_{(i,i),t}$ denotes the $i$-th diagonal element of $S_{t}$, and $\text{Top}(\cdot)$ is a 0-1 indicator function returning if $v_{i}$ is the top-$k$ salient node at time step $t$.

Based on the time-dependent victim set, we conduct adversarial attacks to spatiotemporal traffic forecasting models. Specifically, we first generate perturbed adversarial traffic features based on gradient descent methods. Take the widely used Projected Gradient Descent (PGD) [8] for illustration, we construct Spatiotemporal Projected Gradient Descent (STPGD) as below,

where $\mathbfcal{H^{\prime}}_{t-T+1:t}^{(i-1)}$ is the adversarial traffic state at $i-1$-th iteration in the iterative gradient descent, $\alpha$ is the step size, and $\text{clip}_{\mathbf{X^{\prime}}_{t-T+1:t},\varepsilon}(\cdot)$ is the operation to bound adversarial features in a $\varepsilon$ ball. Note $\mathbf{X^{\prime}}_{t}^{(0)}=\mathbf{\tilde{X}}_{t}$. Instead of perturbing all nodes as in vanilla PGD, we only inject perturbations on selected victim nodes in $S_{t}$. Similarly, we can generate perturbed adversarial traffic features by extending other gradient based methods, such as MIM [9].

In the testing phase, we can inject the adversarial traffic states $\mathbfcal{H^{\prime}}_{t-T+1:t}=\mathbfcal{H}_{t-T+1:t}+\bigtriangleup\mathbfcal{H^{\prime}}_{t-T+1:t}$ to apply adversarial attack, where $\bigtriangleup\mathbfcal{H}_{t}^{\prime}+\mathbfcal{H}_{t}=\left\{(\mathbf{X}_{t}^{\prime}-\mathbf{X}_{t})\cdot S_{t}+\mathbf{X}_{t},\mathcal{G}_{t}\right\}\in\mathbfcal{H^{\prime}}_{t-T+1:t}$ and $\bigtriangleup\mathbfcal{H^{\prime}}_{t}=\left\{((\mathbf{X^{\prime}}_{t}-\mathbf{X}_{t})\cdot S_{t},0):\left\|S_{t}\right\|_{0}\leq\eta,\left\|(\mathbf{X^{\prime}}_{t}-\mathbf{X}_{t})\cdot S_{t}\right\|_{p}\leq\varepsilon\right\}\in\bigtriangleup\mathbfcal{H^{\prime}}_{t-T+1:t}$. The details of the adversarial spatiotemporal attack framework under the grey-box setting is in algorithm 1.

The overall adversarial spatiotemporal attack can be easily extended to the white-box and black-box settings, which are detailed below.

White-box attack. Since the adversaries can fully access the data and labels under the white-box setting, we directly use the real ground truth traffic states to guide the generation of adversarial traffic states. The detailed algorithm is introduced in Appendix A.1.

Black-box attack. The most restrictive black-box setting assumes limited accessibility to the target model and labels. Therefore, we first employ a surrogate model, which can be learned from the training data or by querying the traffic forecasting service [16, 17]. Then we generate adversarial traffic states based on the surrogate model to attack the targeted traffic forecasting model. Please refer to Appendix A.2 for more details.

We conclude this section with the theoretical upper bound analysis of the proposed adversarial attack strategy. In particular, we demonstrate the attack performance against the spatiotemporal traffic forecasting model is related to the number of chosen victim nodes, the budget of adversarial perturbations, as well as the traffic network topology.

Proof. Please refer to Appendix B.

Datasets. We use two popular real-world datasets to demonstrate the effectiveness of the proposed adversarial attack framework. (1) PEMS-BAY [18] traffic dataset is derived from the California Transportation Agencies (CalTrans) Performance Measurement System (PeMS) ranging from January 1, 2017 to May 31, 2017. 325 traffic sensors in the Bay Area collect traffic data every 5 minutes. (2) METR-LA [19] is a traffic speed dataset collected from 207 Los Angeles County roadway sensors. The traffic speed is recorded every 5 minutes and ranges from March 1, 2012 to June 30, 2012. For evaluation, all datasets are chronologically ordered, we take the first 70% for training, the following 10% for validation, and the rest 20% for testing. The statistics of the two datasets are reported in Appendix C.

Baselines. In the current literature, few studies can be directly applied to the real-valued traffic forecasting attack setting. To guarantee the fairness of comparison, we construct two-step baselines as below. For victim node identification, we adopt random selection and use the topology-based methods (i.e., node degree and betweenness centrality [20]) to select victim nodes. We also employ PageRank (PR) [21] as the baseline to decide the set of victim nodes. For adversarial traffic state generation, we adopt two widely used iterative gradient-based methods, PGD [8] and MIM [9], to generate adversarial perturbations. In summary, we construct eight two-step baselines, PGD-Random, PGD-PR, PGD-Centrality, PGD-Degree, MIM-Random, MIM-PR, MIM-Centrality, and MIM-Degree. For instance, PGD-PR indicates first identifying victim nodes with PageRank and then applying adversarial noises with PGD. Depending on the adversarial perturbation method, we compare two variants of our proposed framework, namely STPGD-TDNS and STMIM-TDNS.

Target model. To evaluate the generalization ability of the proposed adversarial attack framework, we adopt the state-of-the-art spatiotemporal traffic forecasting model, GraphWaveNet (Gwnet) [2], as the target model. Evaluation results on more target models are reported in Appendix F.

Evaluation metrics. Our evaluation focus on both the global and local effect of adversarial attacks on spatiotemporal models,

where $\mathcal{L}(\cdot)$ is a user-defined loss function. Different from the majority target of adversarial attacks that are classification models (e.g., adversarial accuracy), traffic forecasting is defined as a regression task. Therefore, we adopt Mean Average Error (MAE) [22] and Root Mean Square Error (RMSE) [23] for evaluation. More specifically, we define Global MAE (G-MAE), Local MAE (L-MAE), Global RMSE (G-RMSE), Local RMSE (L-RMSE) to evaluate the effect of adversarial attacks on traffic forecasting. Please refer to Appendix D for detailed definitions of four metrics.

Implementation details. All experiments are implemented with PyTorch and performed on a Linux server with 4 RTX 3090 GPUs.The traffic speed is normalized to $[0,1]$. The input length $T$ and output length $\tau$ are set to $12$. We select 10% nodes from the whole nodes as the victim nodes, and $\varepsilon$ is set to $0.5$. The batch size $\gamma$ is set to $64$. The iteration $K$ is set to $5$, and the step size $\alpha$ is set to $0.1$.

Table 1 reports the overall attack performance of our proposed approach against the original forecasting model and eight baselines with respect to four metrics. Note larger value indicates better attack performance and worse forecasting accuracy. Specifically, we can make the following observations. First, the adversarial attack can significantly degrade the traffic forecasting performance. For example, our approach achieves $(67.79\%,62.31\%)$ and $(19.88\%,14.55\%)$ global performance degradation compared with the original forecasting results on PeMS-BAY and METR-LA dataset, respectively. Second, our approach achieves the best attack performance against all baselines. In particular, STPGD-TDNS achieves $(15.80\%,15.39\%)$ global performance improvement and $(23.35\%,17.19\%)$ local performance improvement on the PeMS-BAY dataset. Similarly, STMIM-TDNS achieves $(2.44\%,2.00\%)$ global performance improvement and $(11.20\%,2.70\%)$ local performance improvement on the METR-LA dataset. Moreover, we observe STPGD-TDNS and STMIM-TDNS, two variants of our framework, respectively achieve the best attack performance on PeMS-BAY and METR-LA datasets, which further validate the superiority of our framework for flexibly integrate different adversarial perturbation methods. Overall, our adversarial attack framework successfully disrupts the traffic forecasting model to make biased predictions.

Then we conduct ablation study on our adversarial attack framework. Due to page limit, we report the result of STPGD-TDNS on the PeMS-BAY dataset. We consider two variants of our approach: (1) w/o TDNS that randomly choose victim nodes to attack, and (2) w/o STPGD that apply vanilla PGD noise to selected victim nodes. As reported in Table 2, we observe $(3.91\%,6.28\%,2.97\%,3.60\%)$ and $(33.41\%,52.45\%,26.19\%,32.97\%)$ attack performance degradation on four metrics by removing our proposed TDNS and STPGD module, respectively. The above results demonstrate the effectiveness of the two-step framework. Moreover, we observe that the STPGD module plays a more important role in the adversarial spatiotemporal attack.

We further study the parameter sensitivity of the proposed framework, including the number of victim nodes $\eta$, the perturbation budget $\varepsilon$, and the batch size $\gamma$. Due to page limit, we report the result of G-RMSE on the PeMS-BAY dataset. We observe similar results by using other metrics and on the METR-LA dataset. Each time we vary a parameter, we set other parameters to their default values.

Effect of $\eta$. First, we vary the number of victim nodes from $0\%$ to $40\%$. As reported in Figure 2 (a), Our approach achieves the best attack performance with a limited victim node budget, and the advantage decrease when the attack can be applied to more nodes.

Effect of $\varepsilon$. Second, we vary the perturbation budget from $0\%$ to $90\%$. As shown in Figure 2 (b), the G-RMSE first increase and then slightly decrease. This is perhaps because the clip function in Equation 11 weakens the diversity of attack noises.

Effect of $\gamma$. Finally, we vary the batch size from $8$ to $128$, as illustrated in Figure 2 (c). We observe the adversarial attack is relatively stable to the batch size. However, too large batch size reduces the attack performance, which may induce over smooth of Equation 8.

Table 3 reports the overall attack performance of our proposed approach against the original forecasting model and four PGD-based baselines under the white-box and black-box attack settings. For the white-box attack, since the attacker can fully access the data and model, we re-train the forecasting model without requiring estimating the latest traffic states. For the black-box attack, we adopt STAWNET [12] as the surrogate model. The experimental results are summarized in Table 3. First, we observe adversarial attacks significantly degrade the performance of the traffic forecasting model under both white-box and black-box settings. For examples, our approach achieves $((68.65\%,66.12\%)$ and $(56.67\%,50.78\%)$ global performance degradation compared with the vanilla forecasting model under white-box and black-box attack. Moreover, our approach consistently achieves the best attack performance against baselines. To be more specific, our approach yield $(4.61\%,9.13\%)$ and $(1.70\%,3.28\%)$ global performance improvement under the white-box setting and black-box setting, respectively. In addition, we observe higher attack effectiveness under the white-box setting and lower attack effectiveness under the black-box setting compared to the grey-box setting. This makes sense as the white-box setting can fully access the data and label, while the black-box has more restrictive data accessibility and relies on the surrogate model to apply adversarial spatiotemporal attack.

Finally, we study the defense of adversarial spatiotemporal attacks. One primary goal of our study is to help improve the robustness of spatiotemporal forecasting models. Therefore, we propose to incorporate the adversarial training scheme for traffic forecasting models with our adversarial traffic states, denoted by AT-TNDS. We compare it with (1) conventional adversarial training (AT) [8] and (2) Mixup [24] with our adversarial traffic states. Note that we also tried other strategies, such as adding $L_{2}$ regularization, etc., which fail to defend the adversarial spatiotemporal attack. The other state-of-the-art adversarial training methods, such as TRADE [25], cannot be directly applied in regression tasks. Please refer to Appendix E for more training details.

The results in G-MAE on the PeMS-BAY are reported in Table 4. Overall, we observe AT or Mixup can successfully resist the adversarial spatiotemporal attack, and AT-TDNS that combines the adversarial training scheme with our adversarial traffic states achieves the best defensive performance. The above results indicate the defensibility of adversarial spatiotemporal attacks, which should be further investigated to deliver a more reliable spatiotemporal forecasting service in the future.