Poisoning Network Flow Classifiers

Giorgio Severi Simona Boboila Alina Oprea

Northeastern University Northeastern University Northeastern University
[email protected]

John Holodnak Kendra Kratkiewicz Jason Matterer

MIT Lincoln Laboratory MIT Lincoln Laboratory STR∗

ABSTRACT The same conditions that spurred the development of new auto-
As machine learning (ML) classifiers increasingly oversee the au- mated network traffic analysis systems, have also led researchers
tomated monitoring of network traffic, studying their resilience to develop adversarial machine learning attacks against them, tar-
against adversarial attacks becomes critical. This paper focuses on geting both deployed models [5, 9, 13, 25, 64] (evasion attacks) and,
poisoning attacks, specifically backdoor attacks, against network albeit to a lesser extent, their training process [4, 30, 41, 58] (poi-
traffic flow classifiers. We investigate the challenging scenario of soning attacks). We believe this second category is particularly
clean-label poisoning where the adversary’s capabilities are con- interesting, both from an academic perspective as well as a practi-
strained to tampering only with the training data — without the cal one. Recent research on perceived security risks of companies
ability to arbitrarily modify the training labels or any other compo- deploying machine learning models repeatedly highlighted poison-
nent of the training process. We describe a trigger crafting strategy ing attacks as a critical threat to operational ML systems [22, 78].
that leverages model interpretability techniques to generate trigger Yet, much of the prior research on poisoning attacks in this domain
patterns that are effective even at very low poisoning rates. Finally, tends to adopt threat models primarily formulated in the sphere of
we design novel strategies to generate stealthy triggers, including image classification, such as assuming that the victim would accept
an approach based on generative Bayesian network models, with a pre-trained model from a third party [58], thus allowing adversar-
the goal of minimizing the conspicuousness of the trigger, and thus ial control over the entire training phase, or granting the adversary
making detection of an ongoing poisoning campaign more chal- the ability to tamper with the training labels [4]. As awareness of
lenging. Our findings provide significant insights into the feasibility poisoning attacks permeates more extensively, it is reasonable to as-
of poisoning attacks on network traffic classifiers used in multi- sume that companies developing these types of systems will exhibit
ple scenarios, including detecting malicious communication and an increased wariness to trust third parties providing pre-trained
application classification. classifiers, and will likely spend resources and effort to control
or vet both code and infrastructure used during training. For this
ACM Reference Format: reason, we believe it is particularly interesting to focus on the less
Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz,
studied scenario of an adversary who is restricted to tampering
and Jason Matterer. 2023. Poisoning Network Flow Classifiers . In An-
only with the training data (data-only attack) by disseminating a
nual Computer Security Applications Conference (ACSAC ’23), December
04–08, 2023, Austin, TX, USA. ACM, New York, NY, USA, 15 pages. https: small quantity of maliciously crafted points, and without the ability
//doi.org/10.1145/3627106.3627123 to modify the labels assigned to training data (clean-label) or any
other component of the training process.
Our aim is to investigate the feasibility and effects of poison-
1 INTRODUCTION
ing attacks on network traffic flow classifiers, and in particular
Automated monitoring of network traffic plays a critical role in backdoor attacks —where an association is induced between a trig-
the security posture of many companies and institutions. The large ger pattern and an adversarially chosen output of the model. Our
volumes of data involved, and the necessity for rapid decision- approach focuses on the manipulation of aggregated traffic flow
making, have led to solutions that increasingly rely on machine features rather than packet-level content, as they are common in
learning (ML) classifiers to provide timely warnings of potentially traffic classification applications [53, 61, 88]. We will focus on sys-
malicious behaviors on the network. Given the relevance of this tems that compute aggregated features starting from the outputs
task, undiminished despite being studied for quite a long time [54], of the network monitoring tool Zeek, because of its large user base.
a number of machine learning based systems have been proposed It is important to note that, despite the perceived relevance of poi-
in recent years [29, 52, 60, 61, 88] to classify network traffic. soning attacks, it is often remarkably difficult for an adversary to
∗ Work done while staff member at MIT Lincoln Laboratory.
successfully run a poisoning campaign against classifiers operating
on constraint-heavy tabular data, such as cybersecurity data — like
network flows or malware samples [73]. This is a well known issue
Publication rights licensed to ACM. ACM acknowledges that this contribution was
authored or co-authored by an employee, contractor or affiliate of the United States in adversarial ML, illustrated in detail by [68] and often referred to
government. As such, the Government retains a nonexclusive, royalty-free right to as problem-space mapping. It stems from the complexity of craft-
publish or reproduce this article, or to allow others to do so, for Government purposes ing perturbations of the data points (in feature space) that induce
only.
ACSAC ’23, December 04–08, 2023, Austin, TX, USA the desired behavior in the victim model without damaging the
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM. structure of the underlying data object (problem space) necessary
ACM ISBN 979-8-4007-0886-2/23/12. . . $15.00 for it to be generated, parsed, or executed correctly. When dealing
https://doi.org/10.1145/3627106.3627123

337
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

with aggregated network flow data, these difficulties compound pixels (the trigger pattern) was added to a subset of images at train-
with the inherent complexity of handling multivariate tabular data ing time together with an altered label, to induce the prediction of a
consisting of heterogeneous fields. To address these challenges, we target class. Subsequently, Turner et al. [82] and Shafahi et al. [74]
design a novel methodology based on ML explanation methods devised clean-label backdoor attacks which require more poisoning
to determine important features for backdoor creation, and map data samples to be effective, but relax some strong assumptions of
them back into the problem space. Our methods handle complex previous threat models, making them significantly more applicable
dependencies in feature space, generalize to different models and in security scenarios.
feature representations, are effective at low poisoning rates (as low In cybersecurity, the earliest poisoning attacks were designed
as 0.1%), and generate stealthy poisoning attacks. against worm signature generation [57, 66] and spam detectors [56].
In summary, we make the following contributions: (i) We develop More recently, a few studies have looked at packet-level poisoning
a new strategy to craft clean-label, data-only, backdoor poisoning via padding [30, 58], feature-space poisoning in intrusion detec-
attacks against network traffic classifiers that are effective at low tion [4, 42], and label flipping attacks for IoT [65]. Severi et al. [73]
poisoning rates. (ii) We show that our poisoning attacks work across proposed to use model interpretation techniques to generate clean-
different model types, classification tasks, and feature representa- label poisoning attacks against malware classifiers. Their strategies
tions, and we comprehensively evaluate the techniques on several are applicable to security datasets whose records are independent
network traffic datasets used for malware detection and application such as individual files or Android applications, which present a di-
classification. (iii) We propose different strategies, including gener- rect mapping from feature space to problem space. In contrast, our
ative approaches based on Bayesian networks, to make the attacks study explores attacks trained on network traffic, where multiple
inconspicuous and blend the poisoned data with the underlying sequential connections are translated into one single feature-space
training set. To ensure reproducibility, we evaluate our techniques data point; in this setting, inverting triggers from feature to problem
on publicly available datasets, and release all the code used to run space becomes particularly difficult due to data dependencies.
the experiments in the paper1 .
Model Interpretation Techniques. With the proliferation and in-
crease in complexity of ML models, the field of explainable machine
2 BACKGROUND AND RELATED WORK learning, focused on understanding and interpreting model predic-
Machine Learning for Threat Detection. Machine learning tions, has seen a substantial increase in popularity over recent years.
methods have been successfully used to detect several cyber security We are particularly interested in model-agnostic interpretability
threats, including: malicious domains [2, 3, 59, 62, 69], command- techniques, which can be applied to any model. Linardatos et al. [44]
and-control communication between attackers and compromised provide a comprehensive taxonomy of these methods, and conclude
hosts [55, 62], or malicious binaries used by adversaries for distribut- that, among the black-box techniques presented, Shapley Additive
ing exploit code and botnet commands [33, 79]. Several endpoint explanations (SHAP) [48, 49] is the most complete, providing ex-
protection products [31, 32, 50, 51] are now integrating ML tools planations for any model and data type both at a global and local
to proactively detect the rapidly increasing number of threats. scale. SHAP is a game-theory inspired method, which attempts
to quantify how important each feature is for a classifier’s predic-
Adversarial Machine Learning. We can identify two major cate- tions. SHAP improves on other model interpretation techniques
gories of integrity attacks against ML classifiers: (1) evasion attacks, like LIME [72], DeepLIFT [77] and Layer-Wise Relevance Propaga-
which occur at test time and consist in applying an imperceptible tion [6], by introducing a unified measure of feature importance
perturbation to test samples in order to have them misclassified, and that is able to differentiate better among output classes.
(2) poisoning attacks, which influence the training process (either In this study, we also experiment with Gini index [21] and infor-
through tampering with the training dataset or by modifying other mation gain [38, 40] – two of the most popular splitting algorithms
components of the training procedure) to induce wrong predictions in decision trees. A decision tree is built recursively, by choosing at
during inference. For details on other adversarial ML techniques, we each step the feature that provides the best split. Thus, the tree offers
direct the reader to the standardized taxonomy presented in [63]. a natural interpretability, and a straightforward way to compute
In this study, we are focusing on backdoor poisoning attacks, a the importance of each feature towards the model’s predictions.
particularly insidious technique in which the attacker forces the
learner to associate a specific pattern to a desired target objective — Preserving Domain Constraints. Functionality-preserving at-
usually the benign class in cybersecurity applications. While back- tacks on network traffic have mostly looked at evasion during test
door poisoning does not impact the model’s performance on typical time, rather than poisoning. For instance, Wu et al. [84] proposed
test data, it leads to misclassification of test samples that present a packet-level evasion attack against botnet detection, using rein-
the adversarial pattern. ML poisoning has become a top concern forcement learning to guide updates to adversarial samples in a way
in industry [78]. In contrast to evasion attacks which need to gen- that maintains the original functionality. Sheatsley et al. [76] study
erate per-sample perturbations, backdoor triggers, once learned, the challenges associated with the generation of valid adversarial
are powerful and universal as they can be applied to any samples examples that abide domain constraints and develop techniques to
during inference to alter their prediction. learn these constraints from data. Chernikova et al. [13] design eva-
Backdoor poisoning attacks against modern ML models were in- sion attacks against neural networks in constrained environments,
troduced by Gu et al. [23] in BadNets, where a small patch of bright using an iterative optimization method based on gradient descent to
ensure valid numerical domain values. With our constraint-aware
1 https://github.com/ClonedOne/poisoning_network_flow_classifiers

338
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

problem-space mapping, which also takes into account dependen- explanation technique to compute feature importance coefficients,
cies in network traffic, we delve one step further into the challeng- but it prevents any form of inspection of model weights or hid-
ing issue of designing functionality-preserving attacks. den states. This scenario is very common for deployed models, as
Significant advances have been made recently with respect to they often undergo periodical re-training but are only accessible
generating multivariate data. Modern tabular data synthesizers behind controlled APIs. Interacting with a victim system, however,
of mixed data types leverage the power of generative adversarial always imposes a cost on the attacker, whether in terms of actual
networks [11, 18, 19, 86, 91] and diffusion models [39] to create monetary expenses for API quotas, or by increasing the risk of
realistic content from the same distribution as the original data. being discovered. Motivated by this observation, we also explore
Among the different frameworks, FakeTables [11] is the only at- the use of model interpretation methods that do not require any
tempt at preserving functional dependencies in relational tables. access to the classifier, but instead leverage proxy models on local
However, its evaluation is limited to Census and Air Carrier Statis- data (i.e., information gain and Gini coefficients), and can be used
tics datasets, and its ability to capture more complex relationships even when the model is not subject to re-training cycles. Several
between variables is unclear. previous studies on training time attacks [47, 58] relax the model
In this work, we model conditional dependencies in the traf- access constraints, assuming an adversary can train a ML classifier
fic using Bayesian networks – a common choice for generating and provide it to the victim through third-party platforms such as
synthetic relational tables [15, 27, 36, 70, 90]. Bayesian networks Machine Learning as a Service (MLaaS) [71]. However, we believe
offer increased transparency and computational efficiency over that this threat model is rapidly becoming obsolete, at least in the
more complex generative models like generative adversarial net- cybersecurity domain, due to the push for stricter cyber hygiene
works [36]. We believe this is an important advantage in our setting, practices from security vendors, including the reluctance to trust
which deals with large volumes of network traffic featuring multi- third-party model providers and MLaaS platforms [1, 67].
ple variables (e.g., log fields). In cybersecurity, Bayesian networks Importantly, our threat model requires the adversary to have a
have also been used to learn traffic patterns and flag potentially small footprint within the victim network. In practice, the attack
malicious events in intrusion detection systems [16, 34, 83, 85]. could be run by controlling even a single internal host and some
external IPs. . The crafted trigger models a specific traffic pattern
3 THREAT MODEL in a time window, independent of IP values.

Adversary’s Capabilities. Recent work analysing the training Adversary’s Objective. The main objective of the adversary is
time robustness of malware classifiers [73, 89] pointed out that to acquire the ability to consistently trigger desired behavior, or
the use of ever larger quantities of data to train effective security output, from the victim model, after the latter has been trained
classifiers inherently opens up the doors to data-only poisoning on the poisoned data. In this study, we focus on the binary class
attacks, especially in their more stealthy clean-label [74, 81] vari- scenario (0/1), where the goal is defined as having points of a chosen
ants where the adversary does not control the label of the poisoned victim class being mis-labeled as belonging to the target class, when
samples. Thus, in this work, we constrain the adversary to clean- carrying a backdoor pattern that does not violate the constraints
label data-only attacks. This type of setup moves beyond the classic of the data domain. For instance, in the benign/malicious case, the
threat model proposed by Gu et al. [23] and adopted by other re- adversary attempts to have malicious data points mis-classified as
search [12, 47, 58], where the adversary was able to tamper with not benign, where “benign” represents the target class.
only the content of the training points but also the corresponding Adversary’s Target. We select two representative ML classifier
ground-truth labels. models as targets for our attacks: Gradient Boosting decision trees,
Network traffic labeling is often done using antivirus tools or and Feed-forward Neural Networks. Both of these models have
external threat services (e.g., Intrusion Detection Systems, VirusTo- been widely-used in intrusion detection for classifying malicious
tal2 , etc.) [24, 62], making label manipulation hard for an adversary. network traffic, with decision trees often preferred in security con-
Hence, clean-label poisoning is a more realistic threat model, where texts due to their easier interpretation [35]. We study two use cases
access to even a single compromised host is enough to carry out of network traffic classifiers: (1) detection of malicious activities,
the attack by injecting the backdoor into the benign traffic, without and (2) application classification.
tampering with the data collection and labeling process. By dissemi-
Data Format. In our threat model, network traffic consists of con-
nating innocuous looking —but adversarially crafted— data, i.e., the
nection logs (“conn.log” files), which are extracted from packet-level
backdoor, the adversary is able to tamper with a small, yet effective,
PCAP files using the Zeek3 monitoring tool. We use a subset of the
percentage of the training set and induce the desired behavior in
Zeek log fields previously used in the literature that are effective
the learned model.
at detecting malicious traffic [61]. The Zeek log fields used in our
To design the trigger, the adversary requires access to a small
study are described in Appendix A, Table 5, and include port, IP
amount of clean labeled data, 𝐷𝑎 , from a similar distribution as
address, protocol, service, timestamp, duration, packets, payload
the victim’s training data 𝐷. 𝐷𝑎 is used for crafting the backdoor
bytes, and connection state. Thus, the input data is tabular and
pattern and it is disjoint from the training and test datasets.
multivariate, consisting of multiple log fields in either numeric for-
We consider an adversary who has query-only access to the ma-
mat (e.g., bytes, packets, etc.) or categorical format (e.g., connection
chine learning classifier. This allows the attacker to use the SHAP
state, protocol, etc.). A data point in this domain is represented by
2 https://www.virustotal.com/ 3 https://zeek.org/ Previously known as Bro.

339
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

Feature space Problem space

traffic. (ii) Data semantics and dependencies need to be preserved,
Feature Assignment Prototype Trigger Trigger such as value restrictions on specific fields (e.g., upper/lower bounds
Selection (ideal) (realistic) (actual traffic) Injection
on packet length), feature correlations (e.g., protocols use specific
Methods based on Methods to increase
ports), etc. (iii) The injected pattern needs to handle multiple data
model interpretability: stealthiness: types, i.e., numeric and categorical.
• SHAP • Trigger reduction
• Gini coefficients • Trigger generation using
• Inf. Gain (Entropy) Bayesian Networks 4.1 Crafting the Poisoning Data
To address these challenges, we design a novel methodology that
Figure 1: Pipeline for poisoning network flow classifiers. leverages insights from explanation-based methods to determine im-
portant features in feature space, then map them back to constraint-
a sequence of raw log records grouped together. This problem-space aware triggers in problem space. The mapping can be done via: (i)
data point is mapped into a corresponding feature-space data point poisoning attacks using connections directly extracted from ma-
through various aggregation techniques applied over the log field licious traffic; (ii) poisoning attacks with reduced footprint; (iii)
values. generative Bayesian models to increase attack stealthiness.
Our attack strategy, illustrated in Figure 1, consists of five main
Feature Representation. We study two standard and widely
phases:
adopted feature mapping techniques: (1) aggregation, to produce
statistical features, and (2) embeddings— using auto-encoders to au- (I) Select a subset of features that are most important for the
tomatically generate feature vectors. Traffic statistics have multiple class that the adversary wishes to misclassify using model
applications in network monitoring and security [53, 88], which explanation techniques;
require dealing with large volumes of data. For instance, distinct (II) Find an ideal trigger in feature space — we call this an as-
count metrics are used to identify scanning attacks, while volume signment;
metrics or traffic distributions over port numbers and IP address (III) Find a data point that best approximates the ideal trigger
ranges are utilized in anomaly detection [8]. We use aggregation values — this will be our prototype trigger;
methods similar to previous works [8, 61], to derive statistics of (IV) Identify a set of real connections that induce the values
connections. The statistical features used in our study are described observed in the prototype — this set of connections will be
in Appendix A, Table 6, and include traffic volume by internal IP our actual trigger;
(in bytes and packets) within a 30-sec time window, connection (V) Inject the trigger in points of the target class, potentially
counts by transport protocol, connection counts by state, etc. trying to minimize its conspicuousness.
Recent literature also features a variety of approaches for net- Phase I. We first identify the most relevant features for the class
work traffic classification based on auto-encoders [14, 26, 52, 88]. to be misclassified. Our goal is to leverage highly informative fea-
Auto-encoders are unsupervised models that learn to reconstruct tures to coerce the model into associating the trigger pattern with
the training data. They are often used either for anomaly detection the target class. There are a variety of techniques from the field of
or to learn high level features to use in downstream classifiers. model interpretability used to estimate the effect of specific features
Similar to existing work in poisoning attacks and defenses, we towards the classifier’s decision.
assume the feature engineering process is known to the adversary. We start by adapting the SHAP-based technique from [73] to
Removing this assumption and understanding transferability under the network domain. Here, SHAP values are computed for a subset
different feature representations is an interesting open question. of points in 𝐷𝑎 , and their contributions summed per-feature, to
identify the ones most contributing to each class. This approach
4 ATTACK STRATEGY has the advantage of being model agnostic, allowing us to esti-
The formulation of an appropriate trigger pattern is a fundamen- mate feature importance coefficients for any possible victim model.
tal aspect of backdoor poisoning attacks The inherent intricacies Unfortunately, it also assumes the adversary is able to perform a
of network traffic —feature dependencies, multiple data modalities— possibly large number of queries against the victim model.
makes it particularly challenging to ensure that the trigger is mapped To address this potential limitation, we also evaluate the effect
correctly to realizable actions in problem space [68]. This is a stark of selecting the important features through more indirect ways. In
difference with the image domain, where the backdoor trigger can particular we can leverage the information gain and Gini coefficient
be extremely simplistic, such as a bright colored square [23]. metrics used in training decision trees, to estimate the global con-
There are three key requirements that characterize a feasible tributions of each feature. We report a list of the most commonly
poisoning attack: (i) To be effective, the trigger should be easy to selected features across our experiments in Appendix C Table 7.
associate to the target class by the victim model. (ii) The injected The attentive reader will notice here that the approaches we
pattern should appear inconspicuous, so as to avoid detection by mentioned to estimate feature importance are quite different. This
potential human or automated observers. (iii) The perturbations is intentional, and it highlights the modularity of this component.
induced by the injection of the trigger pattern should not affect As long as the adversary is capable of obtaining global estimates of
data validity. While the first two requirements are generic to any feature importance scores, they can use them to guide the attack.
backdoor attack, the third one translates to additional constraints Moreover, with potential future discoveries in the, extremely ac-
on adversarial actions in the network domain, specifically: (i) The tive, field of model interpretation, novel methods could be used to
adversary can only insert traffic, but not modify or remove existing improve the effectiveness of this attack.

340
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

Phase II. Once the subset of important features is selected, we corresponding to non-selected features in the backdoor to make
can proceed to find a suitable assignment of values. To be consistent them appear closer to values common in the target-class natural
with real traffic constraints, we need to ensure that the values data ∈ 𝐷𝑎 . Note that fields influencing the selected (important) fea-
that we select represent information that can be easily added to tures will not be modified, because they carry the backdoor pattern
data points of the non-target class, by injecting new connections, associated with the target class. Our generative approach leverages
without having to remove existing connections. Our features are Bayesian networks, a widely-used probabilistic graphical model
mainly count-based, hence injecting the trigger will increase feature for encoding conditional dependencies among a set of variables,
values. Thus, for the assignment, we select values that correspond to and deriving realistic samples of data [15, 27, 70]. Bayesian net-
the top 𝑡 𝑡ℎ percentile of the corresponding features for non-target works consist of two parts: (1) structure – a directed acyclic graph
class points. Choosing a high percentile is a reasonable heuristic, (DAG) that expresses dependencies among the random variables
as it provides a strong signal. In practice, setting this parameter to associated with the nodes, and (2) parameters – represented by
the 95𝑡ℎ percentile performed well in our experiments. conditional probability distributions associated with each node.

Phase III. Armed with the desired assignment for the selected Structure. Given our objective to synthesize realistic log connec-
features, we can proceed to identify an existing data point that ap- tions (in problem space) that lead to the feature-space prototype,
proximates these ideal trigger values. To find it, in our first attack we we construct a directed acyclic graph 𝐺 = (𝑉 , 𝐸) where the nodes
leverage a mimicry method to scan the non-target (e.g., malicious) 𝑥𝑖 ∈ 𝑉 correspond to fields of interest in the connection log and
class samples and isolate the one with the lowest Euclidean distance the edges 𝑒𝑖 𝑗 ∈ 𝐸 model the inter-dependencies between them. We
from the assignment, in the subspace of the selected features. We explore field-level correlations in connection logs using two statis-
call this point in feature space the trigger prototype. An example tical methods that have been previously used to study the degree of
of the trigger prototype in feature space is given in Appendix C association between variables [37]: the correlation matrix and the
Table 8. pairwise normalized mutual information. In our experiments, both
methods discover similar relationships in 𝐷𝑎 , with the mutual infor-
Phase IV. Up until this point, the process was working completely mation approach bringing out additional inter-dependencies. Note
in feature space. Our explicit goal, however, is to run the attack in that we are not interested in the actual coefficients, rather, in the
problem space. So the next step in the attack chain is to identify, in associational relationships between variables. Thus, we extract the
the attacker’s dataset, a contiguous subset of log connections that strongest pairwise associations, and use them in addition to domain
best approximate the prototype. Enforcing that the selected subset is expertise to guide the design of the DAG structure. For instance,
contiguous ensures that temporal dependencies across log records there is a strong relationship between the number of response
are preserved. This subset of connections represents the actual packets and source packets (resp_pkts ↔ orig_pkts); between the
trigger that we will use to poison the target-class training data. protocol and the response port (proto ↔ resp_p); between the
Appendix C Table 9 shows an excerpt from a trigger materialized connection state and protocol (conn_state ↔ proto), etc.
as a pattern of connections. There is a large body of literature on learning the DAG structure
Phase V. Finally, it is time to inject the trigger in the training directly from data. We point the interested reader to a recent survey
data. This step is quite straightforward, as it the adversary is in by Kitson et al. [37]. However, computing the graphical structure
control of generating the poisoned data, and can execute the trigger remains a major challenge, as this is an NP-hard problem, where
connections in the specified order. We next describe two strategies the solution space grows super-exponentially with the number of
for increasing trigger stealthiness before injection. variables. Resorting to a hybrid approach [37] that incorporates
expert knowledge is a common practice that alleviates this issue.
4.2 Increasing Attack Stealthiness The survey also highlights the additional complexity in modeling
the DAG when continuous variables are parents of discrete ones,
Beyond the basic objective of maximizing attack success, the ad- and when there are more than two dependency levels in the graph.
versary may have the additional goal of minimizing the chance of Based on the above considerations, we design the directed acyclic
being detected. To achieve this secondary goal, the adversary may graph presented in Figure 2. For practical reasons, we filter out
wish to slightly alter the trigger before injecting it in the training some associations that incur a high complexity when modeling the
data. In particular, we study two strategies: (1) trigger size reduction conditional probability distributions. To ensure that the generated
and (2) trigger generation using Bayesian models. traffic still reflects the inter-dependency patterns seen in the data,
Trigger size reduction. The first strategy consists of minimizing we inspect the poisoned training dataset using the same statistical
the trigger footprint, by removing all the connections that are not techniques (correlation matrix and mutual information). We include
strictly necessary to achieve the values specified in the prototype the mutual information matrix on the clean adversarial dataset
for the subset of important features (such as connections on other (Appendix E, Figure 8a) and on the training dataset poisoned with
ports). We then select the smallest subset of contiguous connections the Generated trigger method (Appendix E, Figure 8b), to show
that would produce the desired values for the selected features. that the associational relationships between variables are preserved
after poisoning (though the actual coefficients may vary).
Trigger generation using Bayesian networks. The second strat-
egy aims at reducing the conspicuousness of the trigger by blending Parameters. Bayesian networks follow the local Markov prop-
it with the set of connections underlying the data point where it erty, where the probability distribution of each node, modeled as a
is embedded. To this end, we generate the values of the log fields random variable 𝑥𝑖 , depends only on the probability distributions

341
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

resp_p orig_pkts The Neris botnet scenario unfolds over three capture periods. We
use two of these periods for training our models, and we partition
service proto orig_p resp_pkts orig_bytes the last one in two subsets, keeping 85% of the connections for the
test set, and 15% for the adversarial set, 𝐷𝑎 .
conn_state resp_bytes
CIC IDS 2018 Botnet: From CTU-13, we moved to a recent dataset
for intrusion detection systcheems, the Canadian Institute for Cy-
Figure 2: Directed Acyclic Graph (DAG) representing the bersecurity (CIC) IDS 2018 dataset [75]. We experimented with the
inter-dependencies between log connection fields. botnet scenario, in which the adversary uses the Zeus and Ares mal-
ware packages to infect victim machines and perform exfiltration
of its parents. Thus, the joint probability distribution of a Bayesian actions. This dataset includes a mixture of malicious and benign
network consisting of 𝑛 nodes is represented as: 𝑝 (𝑥 1, 𝑥 2, · · · , 𝑥𝑛 ) = samples and is also heavily imbalanced.
Î𝑛
𝑖=1 𝑝 (𝑥𝑖 |𝑥 𝑃𝑖 ), where 𝑃𝑖 is the set of parents for node 𝑖, and the CIC ISCX 2016 dataset: This dataset contains several application
conditional probability of node 𝑖 is expressed as 𝑝 (𝑥𝑖 |𝑥 𝑃𝑖 ). traffic categories, such as chat, video, and file transfer. We leverage
Sampling. The DAG is traversed in a hierarchical manner, one the CIC ISCX 2016 dataset [17] to explore another scenario where
step at a time, as a sequential decision problem based on probabili- an adversary may affect the outcome via poisoning: detection of
ties derived from the data, with the goal of generating a realistic banned applications. For instance, to comply with company policies,
set of field-value assignments. The value assignments for nodes an organization monitors its internal network to identify usage
at the top of the hierarchy are sampled independently, from the of prohibited applications. An adversary may attempt to disguise
corresponding probability distribution, while the nodes on lower traffic originating from a banned application as another type of
levels are conditioned on parent values during sampling. We com- traffic. We study two examples of classification tasks on the non-vpn
pute the conditional probabilities of categorical fields (e.g., ports, traffic of this dataset: (1) File vs Video, where we induce the learner
service, protocol, connection state), and model numerical fields to mistake video traffic flows as file transfer, and (2) Chat vs Video,
(e.g., originator/responder packets and bytes) through Gaussian where the classifier mis-labels video traffic as chat communication.
kernel density estimation (KDE). An example of the KDE learned Performance Metrics. Similar to previous work in this area [58,
from the data, and used to estimate the number of exchanged bytes 73], we are interested in the following indicators of performance
between a source (originator) and a destination (responder), given for the backdoored model:
the number of packets, is presented in Appendix D, Figure 7.
• Attack Success Rate (ASR). This is the fraction of test data
Given the complexity of sampling from hybrid Bayesian net-
points which are mis-classified as belonging to the target
works, we approximate the conditional sampling process with a
class. We evaluate this metric on a subset of points that have
heuristic, described in Table 1. We consider an example where the
been previously correctly classified by a clean model trained
log fields corresponding to the most important features have been
with the same original training data and random seed.
set to the TCP protocol and responder port 80. Our generative
method synthesizes values for the rest of the fields, in an attempt • Performance degradation on clean data. This metric captures
to make the trigger blend in with the target class. We show in our the side effects of poisoning, by evaluating the ability of the
evaluation that the synthesized poisoning traffic is a good approxi- backdoored model to maintain its predictive performance on
𝑝
mation of clean network traffic, both in terms of Jensen-Shannon clean samples. Let 𝐹 1 be the F1 score of the poisoned model
distance between distributions (Section 5.3) and preservation of on the clean test set, and 𝐹 1𝑐 the test score of a non-poisoned
field-level dependencies (Appendix E). model trained equally, the performance degradation on clean
data at runtime is: Δ𝐹 1 = |𝐹 1 − 𝐹 1𝑐 |.
𝑝

5 EXPERIMENTAL RESULTS Unless otherwise noted, all the results shown in the following
sections are averages of five experiments with different random
5.1 Experimental Setup seeds, reported with their relative standard deviations.
In this section, we describe the datasets and performance metrics
used in our evaluation. We also present the baseline performance Parameters. We define 𝑝% as the percentage of feature-space
of the target classifiers (without poisoning). points of the training dataset that have been compromised by an ad-
versary. Since the amount of poisoned points is generally a critical
Datasets. We used three public datasets commonly used in cyberse- parameter of any poisoning attack, we measure the attack perfor-
curity research for intrusion detection and application classification. mance across multiple poison percentage values 𝑝% . At runtime,
we randomly select a subset of test points to inject the trigger.
CTU-13 Neris Botnet: We started our experimentation with the
Specifically, we select 200 points for the CTU-13 and CIC IDS 2018
Neris botnet scenario of the well-known CTU-13 dataset [20]. This
datasets, and 80 for the CIC ISCX 2016 dataset (due its smaller size).
dataset offers a window into the world of botnet traffic, captured
within a university network and featuring a blend of both malicious Baseline Model Performance. As mentioned in our threat model,
and benign traffic. Despite the sizeable number of connections we consider two representative classifiers: a Gradient Boosting
(≈ 9∗106 ), the classes are extremely imbalanced, with a significantly Decision Tree (GB), and a Feed Forward Neural Network (FFNN).
larger number of benign than malicious data points. Note that the Note that we are not interested in finding the most effective possible
class imbalance is a common characteristic of security applications. learner for the classification task at hand, instead our focus is on

342
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

Table 1: Sampling method for each dependency described in the DAG from Figure 2. In this example, we assume that the most
important features correspond to protocol and port; their values (TCP protocol on port 80) have been determined in Phase II of
our strategy. Here, our generative method samples the rest of the log field values. 𝐷𝑎 represents the attacker’s dataset.

Dependency Sampling method

1. resp_p → service Select subset from attacker’s data, 𝐷𝑎 , with resp_p = 80. Sample a value for service (S) according to the observed
probabilities.
2. proto → conn_state Subset 𝐷𝑎 with proto = TCP. Sample conn_state according to the observed probabilities.
3. resp_p → orig_p Subset 𝐷𝑎 with resp_p = 80. Sample orig_p according to the observed probabilities.
4. orig_pkts Sample a value for orig_pkts from the KDE learned on 𝐷𝑎 .
5. orig_pkts → resp_pkts Subset 𝐷𝑎 based on orig_pkts. Learn the KDE for resp_pkts from the subset. Sample resp_pkts from the KDE.
6. orig_pkts → orig_bytes Learn the KDE distribution 𝐷𝑂 of originator bytes-per-packet from 𝐷𝑎 . Given previously sampled value for
number of packets, orig_pkts = 𝑚, sample and sum up 1, · · · , 𝑚 values from the distribution 𝐷𝑂 .
7. resp_pkts → resp_bytes Learn the KDE distribution 𝐷𝑅 of responder bytes-per-packet from 𝐷𝑎 . Given previously sampled value for
number of packets, resp_pkts = 𝑛, sample and sum up 1, · · · , 𝑛 values from the distribution 𝐷𝑅 .

Table 2: Base performance of the classifiers, avg. over 5 runs.

Model Accuracy F1 score Precision Recall

CTU-13 Neris Botnet
GB 0.999 0.959 0.996 0.925
FFNN 0.999 0.927 0.971 0.887
CIC-IDS 2018 Botnet
GB 0.999 0.994 0.993 0.995
FFNN 0.999 0.995 0.999 0.991
ISCX 2016 File/Video
GB 0.962 0.800 0.799 0.802
FFNN 0.941 0.719 0.666 0.780
ISCX 2016 Chat/Video
GB 0.936 0.901 0.928 0.875 (a) Gradient Boosting model
FFNN 0.947 0.919 0.939 0.900

selecting generic and widely adopted classifiers to showcase the

adaptability of our attack strategy. Baseline values for accuracy, F1
score, precision, and recall of the classifiers are reported in Table 2.

5.2 Impact of Feature Selection

Similar to the procedure reported in [73], our initial feature selection
strategy revolved around computing local feature importance scores
with SHAP and then aggregating them to obtain global indicators
for each feature of the magnitude and direction of impact for each
feature. As mentioned in Section 4.1, however, this approach has
an important drawback: it requires to perform a potentially large
number of queries against the victim classifier. To obviate this (b) Feed-forward Neural Network model
issue, we also considered ways in which the adversary can extract
feature importance estimates directly from their data subset, 𝐷𝑎 . Figure 3: Attack success rate (ASR) for the CTU-13 Neris
In practice, we experimented with fitting a Decision Tree on 𝐷𝑎 , Botnet scenario with different models and feature selection
following either the Gini impurity (Gini) or the information gain strategies.
(Entropy) criteria, and using the importance estimate given by the
reduction of the criterion induced by the feature4 .
random. Looking at the features selected by the different strate-
The three feature selection strategies implemented (Entropy,
gies, we generally observe that Entropy and Gini tend to assign
Gini, SHAP) use the top eight most important features to design
scores that are strongly positive only for a very small number of
the trigger pattern, and are compared against Random, a baseline
features (typically 1-3), while SHAP scores are distributed more
strategy that chooses the same number of features uniformly at
evenly. This observation, together with the desire to minimize the
4 Usingthe implementation in Scikit-Learn https://scikit-learn.org/stable/modules/ trigger footprint, informed our decision to select the eight most
generated/sklearn.tree.DecisionTreeClassifier.html relevant features. We also experimented with different values of this

343
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

parameter, halving and doubling the number of selected features, Table 3: Area under the Precision-Recall Curve and F1 score
but we found that eight were sufficient to achieve satisfying ASRs. obtained by performing anomaly detection on the poisoned
Attack Success Rate: We show the results of these experiments data with an Isolation Forest model trained on a clean subset
in Figure 3. On average, we found the Entropy strategy to be the of the training data. CTU-13 Neris, at 1% poisoning rate.
most successful against both classifiers on this dataset. The Random
strategy leads to inconsistent results: occasionally, it stumbles upon Strategy Model Trigger PR AUC 𝐹 1 score
useful features, but overall attacks relying on Random selection Full 0.056 0.013
perform worse than attacks guided by the other feature selection Entropy Any Reduced 0.045 0.012
methods. Figure 3 also illustrates a major finding – our attacks per- Generated 0.078 0.018
form well even at very small poisoning rates such as 0.1%, where Full 0.099 0.015
they reach an attack success rate of up to 0.7 against the Gradient Gradient Boosting Reduced 0.070 0.013
Boosting classifier. As expected, increasing the poisoning percent- Generated 0.099 0.019
SHAP
age leads to an increase in attack success rate; for instance, an ASR Full 0.061 0.015
of 0.95 is obtained with Entropy at 1.0% poisoning. By comparison, Feed-forward NN Reduced 0.047 0.014
previous works only considered larger poisoning rates (e.g, 2% to Generated 0.052 0.012
20% in [41], 20% samples from nine (out of ten) non-target classes
in [58]). We also notice that some of the variance in the ASR re-
sults can be attributed to a somewhat bimodal distribution. This and Reduced trigger deliver an attack success rate of about 0.7 and
can be partially explained with differences in the resulting trigger 0.4, respectively, while the Generative trigger is able to synthesize
sizes, with Figure 4b highlighting the correlation between larger more effective triggers, which leads to attack success rates over 0.7.
triggers and higher ASR. We leave a more detailed analysis of the Figure 4b studies the correlation between trigger size (measured
distribution of the ASR scores for future work. in number of connections) and attack success rate for each type
Furthermore, we observe that the SHAP strategy, while working of trigger. Each data point represented in the figure constitutes a
well in some scenarios (especially for the application classification separate experiment, while the regression lines capture the trend
tasks in Section 5.5) does not, on average, lead to better results (how ASR changes as the trigger size changes). These figures show
than estimating feature importance through proxy models (Entropy that the generative method leads to consistently smaller triggers
and Gini). This makes the attack quite easy to run in practice, as it than the other two methods, without sacrificing attack success. This
circumvents the necessity to run multiple, potentially expensive, result is indicative of the power of generative models in knowledge
queries to the victim model. discovery, and, in our case, their ability to synthesize a small set of
Performance degradation on clean data: While these results show realistic log connections that lead to the feature-space prototype.
that the attack causes the poisoned model to misclassify poisoned Figure 4b also shows that the size reduction strategy is able to create
data, we also want to make sure that the performance on clean triggers (Reduced trigger) that are smaller than the Full trigger, but
data is maintained. The average Δ𝐹 1 across poisoning rates and at the expense of the attack success rate.
feature selection strategies in our experiments was below 0.037,
Evaluation of attack stealthiness in feature space. Next, we
demonstrating that the side effects of the attack are minimal. The
evaluate the attack stealthiness in feature space, using the Isolation
neural network model exhibits on average a slightly larger decrease
Forest [45] algorithm for anomaly detection. The objective of this
when compared against the Gradient Boosting classifier, especially
experiment is to see whether a standard technique for anomaly
when the Entropy and Gini feature selection strategies are used.
detection can identify and flag the poisoned samples as anomalies.
The anomaly detector is trained on a clean subset of data, which is
5.3 Attack Stealthiness completely disjoint from the poisoned data points and consists of
Remaining undetected is an important factor in running a success- 10% of the entire training dataset.
ful poisoning campaign. Here, we study the impact of our two ap- Table 3 presents the anomaly detection results on the poisoned
proaches for increasing attack stealthiness described in Section 4.2: data obtained with each trigger type (Full, Reduced, and Generated).
reducing the trigger size (Reduced trigger) and generating the trig- For comparison, we evaluate both the entropy-based and the SHAP-
ger connections using Bayesian networks (Generated trigger). We based feature selection strategies used to craft the injected pattern.
start by analyzing the attack success with the different types of Since SHAP queries the model to compute feature relevance scores,
triggers, followed by a quantitative comparison of their stealthiness we present the anomaly detection results separately for a SHAP-
in feature space (via anomaly detection), and in problem space (via guided attack against a Gradient Boosting classifier and against a
the Jensen-Shannon distance). Feed-forward Neural Network. Across the board, we observe very
low Precision-Recall area under the curve (AUC) scores (in the 0.045
Evaluation of attack success. Figure 4a shows the attack success
– 0.099 range), as well as very low 𝐹 1 scores (in the 0.012 – 0.019
rate as a function of the poisoning percentage for the three different
range). These results demonstrate the difficulty of differentiating
types of triggers: Full, Reduced, and Generated. We observe that
the poisoned data points from the clean data points, and indicate
all triggers are able to mount effective attacks against the Gradient
that the poisoning attacks are highly inconspicuous in feature space.
Boosting classifier, with attack success rates over 0.8 when 0.5%
or more of the training data is poisoned. The Feed-forward Neural Evaluation of attack stealthiness in problem space. We also
Network is generally more resilient to our attacks: the Full trigger evaluate attack stealthiness in problem space, in terms of how

344
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

(a) Comparison of attack success rates (ASR) as a function of poisoning percentage.

(b) Correlation between the number of connections composing the trigger and the attack success rate (ASR). Each point represents a separate
experiment. Curve fitting illustrating the trend is performed using linear regression.

Figure 4: Analysis of trigger selection strategy. CTU-13 Neris Botnet scenario, with the Entropy feature selection strategy.

close the poisoned data is to the target class, here represented by

the benign class (normal traffic). We leverage the Jensen-Shannon
divergence [43], a normalized and symmetrical scoring method for
measuring the similarity between two probability distributions, and
in particular we use the distance formulation defined as the square
root of the divergence, which is zero for identical distributions. We
compute the distance for each field in the connection logs (e.g.,
bytes, port, connection state, etc.), and report the average across
all fields. As a baseline, we compute the average Jensen-Shannon
distance between the target class points (benign log connections
only) of the training and test datasets, capturing the distribution
shift between train and test data. For the CTU-13 Neris Botnet Figure 5: Jensen-Shannon distance between the poisoned and
dataset, we evaluated this reference distance as being D_ref = clean training dataset, averaged over all considered conn.log
JS(train, test) = 0.24. fields. For reference, the average JS distance value between
Figure 5 shows the Jensen-Shannon distance between the poi- the original training data and test data is 0.24. CTU-13 Neris
soned and clean training dataset for each of the trigger types. The Botnet experiments, at 1% poisoning rate.
figure illustrates that all three strategies produce stealthy attacks,
characterized by average Jensen-Shannon distances that are com- study feature encodings, which are automatically learned with
fortably lower than D_ref. Furthermore, the generative method an auto-encoder architecture. Together with statistical features,
(Generated trigger) constructs the most inconspicuous triggers, encoded features are common in network traffic classification, and
followed by the trigger size reduction method (Reduced trigger). auto-encoder models have been widely adopted for this task by
previous works [14, 26, 52, 88]. To generate these features, we
5.4 Impact of Feature Representation first train an auto-encoder model in an unsupervised manner, with
The feature representation used by the learning task can strongly the goal of minimizing the reconstruction error. Then the encoder
influence the attack success. In the next set of experiments, we portion of the model is run on the same training data to extract the

345
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

Table 4: Results on the CTU-13 Neris Botnet scenario, where

the victim model uses an auto-encoder to learn the feature
representation. Entropy strategy.

Poison budget 0.5% 1% 2% 4% 5% 10%

ASR 0.013 0.066 0.166 0.362 0.406 0.634
Stand. dev. 0.009 0.045 0.134 0.100 0.140 0.109
Δ𝐹 1 Test 0.002 0.003 0.005 0.009 0.011 0.007

high-level features used to train the feed-forward neural network

architecture considered in previous experiments. Since the auto-
encoder requires its inputs to be of a consistent shape, instead of
features extracted from 30-second time windows, here the model
is provided with an input representation consisting of contiguous
blocks of 100 connections. Given that features are extracted from
connection blocks of a fixed size, we also fix the trigger size to be 50
connections long. We found this value empirically by experimenting
with different trigger sizes, and noticed that smaller ones would
lead to unsatisfying attack results. While the trigger is relatively
large compared to the unit block size, it is worth noting that the
total number of connections introduced by the attack is still very
limited when compared to the size of the the training set.
Table 4 reports the mean attack success rate of the Entropy
strategy when applied in this setup, at different poison percentages,
together with its standard deviation across 5 experiments and the
average degradation in performance of the victim model on clean
data. Since the auto-encoder was trained in an unsupervised fashion
to minimize the reconstruction loss, we expect this training loss
to impact negatively the overall success of the attack. In fact, we
do observe a general reduction of the success rate compared to
the simple neural network model, especially for limited poisoning
budgets (≤ 1%). However, if the adversary is allowed to increase
the poisoning rate beyond 1%, we observe that the attack scales
nicely with larger poisoning budgets. At the same time, the Δ𝐹 1
values remain generally low even at larger poison percentages. Figure 6: Attack success rate (ASR) on the CIC IDS 2018 Botnet
and the CIC ISCX 2016 dataset, full trigger.
5.5 Other datasets
In the previous sections, we carried out an in-depth evaluation of Δ𝐹 1 is between 0.002 and 0.046, with the SHAP strategy resulting
various attack characteristics and their impact on the attack success. in slightly larger shifts than the other feature selection methods.
In this section, we investigate how generalizable this poisoning
approach is by testing it on different datasets and other classification 6 DISCUSSION AND LIMITATIONS
tasks. We evaluate here a second cybersecurity task on the CIC Well-formed triggers. The problem-space mapping of the triggers
IDS 2018 dataset, and two application classification scenarios on is a particularly challenging task, owing to its inherent complexity
CIC ISCX 2016. For all of these case studies, we use the statistical on arbitrary networks. For instance, the adversary may experience
features (see Appendix A, Table 6) and the full trigger strategy. a situation where two connection events are inter-dependent, due
We report the attack success rate at different poisoning percent- to the internal state of Zeek, but the trigger does not include both of
ages in Figure 6. Due to the much smaller size of the ISCX dataset, them simultaneously — this could occur if the connections happen
we test up to slightly larger poison percentage values — for instance across the border of two time windows. Inter-dependent connection
in the Chat/Video scenario, 0.1% of the training set would amount events may take place in the case of hosts running the FTP proto-
to a single poisoning point. In general, we observe similar trends col. Documentation on this type of connections for Zeek is quite
as in previous experiments, with the SHAP and Entropy strategies scarce, but a dedicated attacker could allocate time and resources
performing similarly, and achieving significant attack success rates to enumerate all possible corner cases and explicitly avoid them
even with very limited poison budgets. during the trigger creation phase.
We also evaluated the poisoned model on clean test data, to Our generative strategy leads to a high ASR with a small foot-
verify whether the poisoned model is still able to classify clean test print and is applicable to both stateless (UDP) and stateful protocols
data correctly. We obtained very limited reductions in 𝐹 1 scores: (TCP). However, it could generate some connection events that are

346
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

not feasible in practice, particularly for stateful protocols (TCP). be lessened through different strategies to decrease the likelihood
There are two ways to address this potential issue. First, given the of a defender discovering an ongoing poisoning campaign.
relentless pace of improvements in generative models, including Furthermore, we demonstrated that this form of poisoning has a
those targeting tabular data [7, 87], we expect that the ability of relatively wide applicability for various objectives across different
generative models to infer the inter-feature constraints that char- types of classification tasks. The implications of these findings
acterize this data modality will increase significantly in the very extend our understanding of ML security in practical contexts,
short term. In parallel, the adversary could attempt to verify the and prompt further investigation into effective defense strategies
correctness of the generated connections using a model checker against these refined attack methodologies.
and a formal model of the TCP protocol, and simply reject the non-
conforming ones. Both approaches are exciting avenues for future ACKNOWLEDGMENTS
research, and we leave their in-depth analysis to future work.
This research was sponsored by the U.S. Army Combat Capabilities
Labeling. Network traffic labeling usually relies on intrusion de- Development Command Army Research Laboratory (DEVCOM
tection systems, antivirus tools and external threat services [24, 62]. ARL) under Cooperative Agreement Number W911NF-13-2-0045,
In our threat model, the adversary has no control on the labels, and and the Department of Defense Multidisciplinary Research Pro-
simply injects the poisoning traffic into benign connections. Hence, gram of the University Research Initiative (MURI) under contract
the question arises: Will the poisoned samples still have a benign W911NF-21-1-0322.
label? We assume the poisoned samples remain benign, based on DISTRIBUTION STATEMENT A. Approved for public release.
the following reasons: (1) The Jensen-Shannon distance between Distribution is unlimited. This material is based upon work sup-
poisoned and clean samples is very small (Figure 5); (2) Anom- ported by the Under Secretary of Defense for Research and En-
aly detection (Table 3) cannot identify the poisoned samples (F1 gineering under Air Force Contract No. FA8702-15-D-0001. Any
score < 0.02); (3) Features are extracted from connections metadata, opinions, findings, conclusions or recommendations expressed in
and the actual packet contents do not need to be malicious. this material are those of the author(s) and do not necessarily re-
Mitigation. We designed methods to hide the poisoning campaign, flect the views of the Under Secretary of Defense for Research and
and showed that our poisoning points are difficult to identify both Engineering.
in feature space, by using anomaly detection techniques, and in
problem space, by analysing the distributional distance of poisoned REFERENCES
data. Defending ML models from backdoor attacks is an open, and [1] Emre Kiciman Andrew Marshall, Jugal Parikh and Ram Shankar Siva Kumar.
extremely complex, research problem. Many of the current pro- 2022. Threat Modeling AI/ML Systems and Dependencies. https://learn.microsoft.
com/en-us/security/engineering/threat-modeling-aiml.
posed solutions are designed to operate in the computer vision [2] Manos Antonakakis, Roberto Perdisci, David Dagon, Wenke Lee, and Nick Feam-
domain [10], or on specific model architectures [46, 80]. In contrast, ster. 2010. Building a Dynamic Reputation System for DNS. In Proceedings of
the 19th USENIX Conference on Security (Washington, DC) (USENIX Security’10).
our attack method generalizes to different model typologies. More- USENIX Association, USA, 18.
over, initial research on defending classifiers from backdoor attacks [3] Manos Antonakakis, Roberto Perdisci, Wenke Lee, Nikolaos Vasiloglou, and
in the security domain [28] highlighted potential trade-offs between David Dagon. 2011. Detecting Malware Domains at the Upper DNS Hierarchy.
In Proceedings of the 20th USENIX Conference on Security (San Francisco, CA)
robustness and utility (e.g., defenses that rely on data sanitization (SEC’11). USENIX Association, USA, 27.
may mistakenly remove a high number of benign samples in an [4] Giovanni Apruzzese, Michele Colajanni, Luca Ferretti, and Mirco Marchetti. 2019.
attempt to prune out potentially poisoned samples). By releasing Addressing Adversarial Attacks Against Security Systems Based on Machine
Learning. In 2019 11th International Conference on Cyber Conflict (CyCon), Vol. 900.
new attack strategies, we hope to encourage future research in 1–18. https://doi.org/10.23919/CYCON.2019.8756865
the challenging direction of defending against backdoor attacks on [5] Md. Ahsan Ayub, William A. Johnson, Douglas A. Talbert, and Ambareen Siraj.
2020. Model Evasion Attack on Intrusion Detection Systems using Adversarial
network traffic. Machine Learning. In 2020 54th Annual Conference on Information Sciences and
Systems (CISS). 1–6. https://doi.org/10.1109/CISS48834.2020.1570617116
[6] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert
7 CONCLUSIONS Müller, and Wojciech Samek. 2016. Layer-wise relevance propagation for neural
networks with local renormalization layers. In Artificial Neural Networks and
With this work we investigated the possibility of carrying out data- Machine Learning–ICANN 2016: 25th International Conference on Artificial Neural
Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25. Springer,
only, clean-label, poisoning attacks against network flow classifiers. 63–71.
We believe this threat model holds substantial significance for the [7] Stavroula Bourou, Andreas El Saer, Terpsichori-Helen Velivassaki, Artemis
security community, due to its closer alignment with the capabilities Voulkidis, and Theodore Zahariadis. 2021. A Review of Tabular Data Syn-
thesis Using GANs on an IDS Dataset. Information 12, 9 (Sept. 2021), 375.
exhibited by sophisticated adversaries observed in the wild, and https://doi.org/10.3390/info12090375
the current best practices in secure ML deployments, in contrast to [8] Martin Burkhart, Mario Strasser, Dilip Many, and Xenofontas Dimitropoulos. 2010.
other prevailing models frequently employed. SEPIA: Privacy-Preserving Aggregation of Multi-Domain Network Events and
Statistics. In 19th USENIX Security Symposium (USENIX Security 10). USENIX Asso-
The attack strategy we introduce can effectively forge consistent ciation, Washington, DC. https://www.usenix.org/conference/usenixsecurity10/
associations between the trigger pattern and the target class even sepia-privacy-preserving-aggregation-multi-domain-network-events-and
[9] Xiaoyu Cao and Neil Zhenqiang Gong. 2017. Mitigating Evasion Attacks to
at extremely low poisoning rates (0.1-0.5% of the training set size). Deep Neural Networks via Region-Based Classification. In Proceedings of the 33rd
This results in notable attack success rates, despite the constrained Annual Computer Security Applications Conference (Orlando, FL, USA) (ACSAC
nature of the attacker. While the attack is effective, it has minimal ’17). Association for Computing Machinery, New York, NY, USA, 278–287. https:
//doi.org/10.1145/3134600.3134606
impacts on the victim model’s generalization abilities when dealing [10] Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Ed-
with clean test data. Additionally, the detectability of the trigger can wards, Taesung Lee, Ian Molloy, and Biplav Srivastava. 2019. Detecting Backdoor

347
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

Attacks on Deep Neural Networks by Activation Clustering. In Workshop on ICDMW58026.2022.00080

Artificial Intelligence Safety. CEUR-WS. [31] IBM. 2023. IBM Security QRadar XDR. https://www.ibm.com/qradar.
[11] Haipeng Chen, Sushil Jajodia, Jing Liu, Noseong Park, Vadim Sokolov, and V. S. [32] Sam Ingalls. 2021. Top XDR Security Solutions for 2022. https://www.
Subrahmanian. 2019. FakeTables: Using GANs to Generate Functional Depen- esecurityplanet.com/products/xdr-security-solutions/.
dency Preserving Tables with Bounded Real Data. In Proceedings of the Twenty- [33] Luca Invernizzi, Sung ju Lee, Stanislav Miskovic, Marco Mellia, Ruben Torres,
Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. Interna- Christopher Kruegel, Sabyasachi Saha, and Giovanni Vigna. 2014. Nazca: Detect-
tional Joint Conferences on Artificial Intelligence Organization, Macao China, ing Malware Distribution in Large-Scale Networks. In NDSS.
2074–2080. https://doi.org/10.24963/ijcai.2019/287 [34] M.A. Jabbar, Rajanikanth Aluvalu, and S. Sai Satyanarayana Reddy. 2017. Intru-
[12] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted sion Detection System Using Bayesian Network and Feature Subset Selection. In
Backdoor Attacks on Deep Learning Systems Using Data Poisoning. CoRR 2017 IEEE International Conference on Computational Intelligence and Computing
abs/1712.05526 (2017). arXiv:1712.05526 http://arxiv.org/abs/1712.05526 Research (ICCIC). 1–5. https://doi.org/10.1109/ICCIC.2017.8524381
[13] Alesia Chernikova and Alina Oprea. 2022. FENCE: Feasible Evasion Attacks on [35] Arthur S. Jacobs, Roman Beltiukov, Walter Willinger, Ronaldo A. Ferreira, Arpit
Neural Networks in Constrained Environments. ACM Trans. Priv. Secur. 25, 4, Gupta, and Lisandro Z. Granville. 2022. AI/ML for Network Security: The Emperor
Article 34 (jul 2022), 34 pages. https://doi.org/10.1145/3544746 Has No Clothes. In Proceedings of the 2022 ACM SIGSAC Conference on Computer
[14] Gianni D’Angelo and Francesco Palmieri. 2021. Network Traffic Classification and Communications Security (Los Angeles, CA, USA) (CCS ’22). Association for
Using Deep Convolutional Recurrent Autoencoder Neural Networks for Spa- Computing Machinery, New York, NY, USA, 1537–1551. https://doi.org/10.1145/
tial–Temporal Features Extraction. Journal of Network and Computer Applications 3548606.3560609
173 (Jan. 2021), 102890. https://doi.org/10.1016/j.jnca.2020.102890 [36] Dhamanpreet Kaur, Matthew Sobiesk, Shubham Patil, Jin Liu, Puran Bhagat,
[15] Tristan Deleu, António Góis, Chris Chinenye Emezue, Mansi Rankawat, Simon Amar Gupta, and Natasha Markuzon. 2020. Application of Bayesian networks
Lacoste-Julien, Stefan Bauer, and Yoshua Bengio. 2022. Bayesian Structure Learn- to generate synthetic health data. Journal of the American Medical Informatics
ing with Generative Flow Networks. In The 38th Conference on Uncertainty in Association 28 (12 2020). https://doi.org/10.1093/jamia/ocaa303
Artificial Intelligence. [37] Neville Kenneth Kitson, Anthony C. Constantinou, Zhigao Guo, Yang Liu, and
[16] Nagaraju Devarakonda, Srinivasulu Pamidi, V. Valli Kumari, and A. Govardhan. Kiattikun Chobtham. 2023. A survey of Bayesian Network structure learning.
2012. Intrusion Detection System using Bayesian Network and Hidden Markov Artificial Intelligence Review (2023). https://doi.org/10.1007/s10462-022-10351-w
Model. Procedia Technology 4 (2012), 506–514. https://doi.org/10.1016/j.protcy. [38] Daphne Koller and Mehran Sahami. 1996. Toward Optimal Feature Selection. In
2012.05.081 2nd International Conference on Computer, Communication, Control Proceedings of the Thirteenth International Conference on International Conference
and Information Technology( C3IT-2012) on February 25 - 26, 2012. on Machine Learning (Bari, Italy) (ICML’96). Morgan Kaufmann Publishers Inc.,
[17] Gerard Draper-Gil, Arash Habibi Lashkari, Mohammad Saiful Islam Mamun, and San Francisco, CA, USA, 284–292.
Ali A. Ghorbani. 2016. Characterization of Encrypted and VPN Traffic Using [39] Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. 2022.
Time-related Features:. In Proceedings of the 2nd International Conference on Infor- TabDDPM: Modelling Tabular Data with Diffusion Models. https://doi.org/10.
mation Systems Security and Privacy. SCITEPRESS - Science and and Technology 48550/arXiv.2209.15421 arXiv:2209.15421 [cs]
Publications, Rome, Italy, 407–414. https://doi.org/10.5220/0005740704070414 [40] Changki Lee and Gary Geunbae Lee. 2006. Information gain and divergence-based
[18] Justin Engelmann and Stefan Lessmann. 2021. Conditional Wasserstein GAN- feature selection for machine learning-based text categorization. Information
based Oversampling of Tabular Data for Imbalanced Learning. Expert Systems Processing & Management 42, 1 (2006), 155–165. https://doi.org/10.1016/j.ipm.
with Applications 174 (01 2021), 114582. https://doi.org/10.1016/j.eswa.2021. 2004.08.006 Formal Methods for Information Retrieval.
114582 [41] Pan Li, Qiang Liu, Wentao Zhao, Dongxu Wang, and Siqi Wang. 2018. Chronic
[19] Ju Fan, Junyou Chen, Tongyu Liu, Yuwei Shen, Guoliang Li, and Xiaoyong Du. Poisoning against Machine Learning Based IDSs Using Edge Pattern Detection.
2020. Relational data synthesis using generative adversarial networks: a design In 2018 IEEE International Conference on Communications (ICC). 1–7. https:
space exploration. Proceedings of the VLDB Endowment 13 (08 2020), 1962–1975. //doi.org/10.1109/ICC.2018.8422328
https://doi.org/10.14778/3407790.3407802 [42] Pan Li, Qiang Liu, Wentao Zhao, Dongxu Wang, and Siqi Wang. 2018. Chronic
[20] S. García, M. Grill, J. Stiborek, and A. Zunino. 2014. An Empirical Comparison Poisoning against Machine Learning Based IDSs Using Edge Pattern Detection.
of Botnet Detection Methods. Computers and Security 45 (Sept. 2014), 100–123. In 2018 IEEE International Conference on Communications (ICC). 1–7. https:
https://doi.org/10.1016/j.cose.2014.05.011 //doi.org/10.1109/ICC.2018.8422328
[21] Joseph Gastwirth. 1972. The Estimation of the Lorenz Curve and Gini Index. The [43] J. Lin. 1991. Divergence measures based on the Shannon entropy. IEEE Transac-
Review of Economics and Statistics 54 (02 1972), 306–16. https://doi.org/10.2307/ tions on Information Theory 37, 1 (1991), 145–151. https://doi.org/10.1109/18.61115
1937992 [44] Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2021. Ex-
[22] Kathrin Grosse, Lukas Bieringer, Tarek R. Besold, Battista Biggio, and Katharina plainable AI: A Review of Machine Learning Interpretability Methods. Entropy
Krombholz. 2023. Machine Learning Security in Industry: A Quantitative Survey. 23, 1 (2021). https://doi.org/10.3390/e23010018
IEEE Transactions on Information Forensics and Security 18 (2023), 1749–1762. [45] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation Forest. In 2008
https://doi.org/10.1109/TIFS.2023.3251842 Eighth IEEE International Conference on Data Mining. IEEE, Pisa, Italy, 413–422.
[23] T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg. 2019. BadNets: Evaluating Backdooring https://doi.org/10.1109/ICDM.2008.17
Attacks on Deep Neural Networks. IEEE Access 7 (2019), 47230–47244. https: [46] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-Pruning: De-
//doi.org/10.1109/ACCESS.2019.2909068 fending Against Backdooring Attacks on Deep Neural Networks. In Research
[24] Jorge Luis Guerra, Carlos Catania, and Eduardo Veas. 2022. Datasets Are Not in Attacks, Intrusions, and Defenses (Lecture Notes in Computer Science), Michael
Enough: Challenges in Labeling Network Traffic. Comput. Secur. 120, C (sep Bailey, Thorsten Holz, Manolis Stamatogiannakis, and Sotiris Ioannidis (Eds.).
2022), 17 pages. https://doi.org/10.1016/j.cose.2022.102810 Springer International Publishing, Cham, 273–294. https://doi.org/10.1007/978-
[25] Mark Handley, Vern Paxson, and Christian Kreibich. 2001. Network Intrusion 3-030-00470-5_13
Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics. [47] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang,
In 10th USENIX Security Symposium (USENIX Security 01). USENIX Association, and Xiangyu Zhang. 2018. Trojaning Attack on Neural Networks. In 25th Annual
Washington, D.C. https://www.usenix.org/conference/10th-usenix-security- Network and Distributed System Security Symposium, NDSS 2018, San Diego, Cali-
symposium/network-intrusion-detection-evasion-traffic-normalization fornia, USA, February 18-21, 2018. The Internet Society. http://wp.internetsociety.
[26] Mingshu He, Xiaojuan Wang, Junhua Zhou, Yuanyuan Xi, Lei Jin, and Xinlei org/ndss/wp-content/uploads/sites/25/2018/02/ndss2018_03A-5_Liu_paper.pdf
Wang. 2021. Deep-Feature-Based Autoencoder Network for Few-Shot Malicious [48] Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin,
Traffic Detection. Security and Communication Networks 2021 (March 2021), Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020.
e6659022. https://doi.org/10.1155/2021/6659022 From local explanations to global understanding with explainable AI for trees.
[27] David Heckerman. 2008. A Tutorial on Learning with Bayesian Networks. Springer Nature Machine Intelligence 2, 1 (2020), 56–67. https://doi.org/10.1038/s42256-
Berlin Heidelberg, Berlin, Heidelberg, 33–82. https://doi.org/10.1007/978-3-540- 019-0138-9
85066-3_3 [49] Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model
[28] Samson Ho, Achyut Reddy, Sridhar Venkatesan, Rauf Izmailov, Ritu Chadha, Predictions. In Advances in Neural Information Processing Systems, I. Guyon,
and Alina Oprea. 2022. Data Sanitization Approach to Mitigate Clean-Label U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett
Attacks Against Malware Detection Systems. In MILCOM 2022 - 2022 IEEE Mili- (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_
tary Communications Conference (MILCOM). 993–998. https://doi.org/10.1109/ files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf
MILCOM55135.2022.10017768 [50] MalwareGuard FireEye 2020. MalwareGuard: FireEye’s Machine Learning Model
[29] Jordan Holland, Paul Schmitt, Prateek Mittal, and Nick Feamster. 2022. Towards to Detect and Prevent Malware. https://www.fireeye.com/blog/products-and-
Reproducible Network Traffic Analysis. arXiv:2203.12410 [cs] services/2018/07/malwareguard-fireeye-machine-learning-model-to-detect-
[30] John T. Holodnak, Olivia Brown, Jason Matterer, and Andrew Lemke. 2022. Back- and-prevent-malware.html.
door Poisoning of Encrypted Traffic Classifiers. In 2022 IEEE International Con- [51] Microsoft. 2021. Microsoft Defender for Endpoint | Microsoft Secu-
ference on Data Mining Workshops (ICDMW). 577–585. https://doi.org/10.1109/ rity. https://www.microsoft.com/en-us/security/business/threat-protection/

348
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

endpoint-defender Machine Learning and Applications (ICMLA). 896–902. https://doi.org/10.1109/

[52] Yisroel Mirsky, Tomer Doitshman, Yuval Elovici, and Asaf Shabtai. 2018. Kit- ICMLA.2015.152
sune: An Ensemble of Autoencoders for Online Network Intrusion Detection. In [72] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should i
Proceedings 2018 Network and Distributed System Security Symposium. Internet trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd
Society, San Diego, CA. https://doi.org/10.14722/ndss.2018.23204 ACM SIGKDD international conference on knowledge discovery and data mining.
[53] Andrew Moore, Denis Zuev, and Michael Crogan. 2005. Discriminators for Use in 1135–1144.
Flow-Based Classification. Technical Report. Queen Mary and Westfield College, [73] Giorgio Severi, Jim Meyer, Scott Coull, and Alina Oprea. 2021. Explanation-
Department of Computer Science. Guided Backdoor Poisoning Attacks Against Malware Classifiers. In 30th USENIX
[54] B. Mukherjee, L.T. Heberlein, and K.N. Levitt. 1994. Network Intrusion Detection. Security Symposium (USENIX Security 21). 1487–1504.
IEEE Network 8, 3 (May 1994), 26–41. https://doi.org/10.1109/65.283931 [74] Ali Shafahi, W. Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer,
[55] Terry Nelms, Roberto Perdisci, and Mustaque Ahamad. 2013. ExecScent: Mining Tudor Dumitras, and Tom Goldstein. 2018. Poison Frogs! Targeted Clean-Label
for New C&C Domains in Live Networks with Adaptive Control Protocol Tem- Poisoning Attacks on Neural Networks. In Advances in Neural Information Pro-
plates. In Proceedings of the 22nd USENIX Conf. on Security. USENIX Association, cessing Systems.
USA, 589–604. [75] Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani. 2018. Toward Gen-
[56] Blaine Nelson, Marco Barreno, Fuching Jack Chi, Anthony D. Joseph, Benjamin erating a New Intrusion Detection Dataset and Intrusion Traffic Characterization:.
I. P. Rubinstein, Udam Saini, Charles Sutton, J. D. Tygar, and Kai Xia. 2008. In Proceedings of the 4th International Conference on Information Systems Secu-
Exploiting Machine Learning to Subvert Your Spam Filter. In Proceedings of the rity and Privacy. SCITEPRESS - Science and Technology Publications, Funchal,
1st Usenix Workshop on Large-Scale Exploits and Emergent Threats (San Francisco, Madeira, Portugal, 108–116. https://doi.org/10.5220/0006639801080116
California) (LEET’08). USENIX Association, USA, Article 7, 9 pages. [76] Ryan Sheatsley, Blaine Hoak, Eric Pauley, Yohan Beugin, Michael J. Weisman, and
[57] James Newsome, Brad Karp, and Dawn Song. 2006. Paragraph: Thwarting Signa- Patrick McDaniel. 2021. On the Robustness of Domain Constraints. In Proceedings
ture Learning by Training Maliciously. In Recent Advances in Intrusion Detection, of the 2021 ACM SIGSAC Conference on Computer and Communications Security
Diego Zamboni and Christopher Kruegel (Eds.). Springer Berlin Heidelberg, (Virtual Event, Republic of Korea) (CCS ’21). Association for Computing Machin-
Berlin, Heidelberg, 81–105. ery, New York, NY, USA, 495–515. https://doi.org/10.1145/3460120.3484570
[58] Rui Ning, Chunsheng Xin, and Hongyi Wu. 2022. TrojanFlow: A Neural Backdoor [77] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning im-
Attack to Deep Learning-based Network Traffic Classifiers. In IEEE INFOCOM portant features through propagating activation differences. In International
2022 - IEEE Conference on Computer Communications. 1429–1438. https://doi. conference on machine learning. PMLR, 3145–3153.
org/10.1109/INFOCOM48880.2022.9796878 [78] Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall,
[59] Talha Ongun, Simona Boboila, Alina Oprea, Tina Eliassi-Rad, Jason Hiser, and Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. 2020. Adver-
Jack W. Davidson. 2022. CELEST: Federated Learning for Globally Coordinated sarial Machine Learning-Industry Perspectives. In 2020 IEEE Security and Privacy
Threat Detection. CoRR abs/2205.11459 (2022). https://doi.org/10.48550/arXiv. Workshops (SPW). 69–75. https://doi.org/10.1109/SPW50608.2020.00028
2205.11459 arXiv:2205.11459 [79] Acar Tamersoy, Kevin Roundy, and Duen Horng Chau. 2014. Guilt by Association:
[60] Talha Ongun, Timothy Sakharaov, Simona Boboila, Alina Oprea, and Tina Eliassi- Large Scale Malware Detection by Mining File-Relation Graphs. In Proceedings
Rad. 2019. On Designing Machine Learning Models for Malicious Network Traffic of the 20th ACM SIGKDD International Conference on Knowledge Discovery and
Classification. arXiv:1907.04846 [cs, stat] Data Mining (New York, New York, USA) (KDD ’14). Association for Computing
[61] Talha Ongun, Oliver Spohngellert, Benjamin Miller, Simona Boboila, Alina Machinery, New York, NY, USA, 1524–1533. https://doi.org/10.1145/2623330.
Oprea, Tina Eliassi-Rad, Jason Hiser, Alastair Nottingham, Jack Davidson, and 2623342
Malathi Veeraraghavan. 2021. PORTFILER: Port-Level Network Profiling for Self- [80] Brandon Tran, Jerry Li, and Aleksander Mądry. 2018. Spectral Signatures in
Propagating Malware Detection. In 2021 IEEE Conference on Communications and Backdoor Attacks. In Proceedings of the 32nd International Conference on Neu-
Network Security (CNS). 182–190. https://doi.org/10.1109/CNS53000.2021.9705045 ral Information Processing Systems (NIPS’18). Curran Associates Inc., Montréal,
[62] Alina Oprea, Zhou Li, Robin Norris, and Kevin Bowers. 2018. MADE: Security Canada, 8011–8021.
Analytics for Enterprise Threat Detection. In Proceedings of Annual Computer Se- [81] Alexander Turner, Dimitris Tsipras, and Aleksander Mądry. 2018. Clean-Label
curity Applications Conference (ACSAC). https://doi.org/10.1145/3274694.3274710 Backdoor Attacks. Manuscript submitted for publication (2018), 21.
[63] Alina Oprea and Apostol Vassilev. 2023. Adversarial Machine Learning: A Taxon- [82] Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2019. Label-
omy and Terminology of Attacks and Mitigations (Draft). Technical Report NIST Consistent Backdoor Attacks. arXiv:1912.02771 [stat.ML]
AI 100-2e2023 ipd. National Institute of Standards and Technology. [83] María Vargas Muñoz, Rafael Martínez-Peláez, Pablo Velarde Alvarado, Efraín
[64] Pavlos Papadopoulos, Oliver Thornewill von Essen, Nikolaos Pitropakis, Christos Moreno-Garcia, Deni Torres-Roman, and José Ceballos-Mejia. 2018. Classification
Chrysoulas, Alexios Mylonas, and William J. Buchanan. 2021. Launching Ad- of network anomalies in flow level network traffic using Bayesian networks. In
versarial Attacks against Network Intrusion Detection Systems for IoT. Journal 2018 International Conference on Electronics, Communications and Computers
of Cybersecurity and Privacy 1, 2 (June 2021), 252–273. https://doi.org/10.3390/ (CONIELECOMP). IEEE, 238–243. https://doi.org/10.1109/CONIELECOMP.2018.
jcp1020014 8327205
[65] Pavlos Papadopoulos, Oliver Thornewill von Essen, Nikolaos Pitropakis, Christos [84] Di Wu, Binxing Fang, Junnan Wang, Qixu Liu, and Xiang Cui. 2019. Evading
Chrysoulas, Alexios Mylonas, and William J. Buchanan. 2021. Launching Adver- Machine Learning Botnet Detection Models via Deep Reinforcement Learning.
sarial Attacks against Network Intrusion Detection Systems for IoT. Journal of In ICC 2019 - 2019 IEEE International Conference on Communications (ICC). IEEE,
Cybersecurity and Privacy 1, 2 (2021), 252–273. https://doi.org/10.3390/jcp1020014 Shanghai, China, 1–6. https://doi.org/10.1109/ICC.2019.8761337
[66] R. Perdisci, M. Sharif, P. Fogla, W. Lee, and D. Dagon. 2006. Misleading Worm [85] Jing Xu and Christian R. Shelton. 2010. Intrusion Detection Using Continuous
Signature Generators Using Deliberate Noise Injection. In 2012 IEEE Symposium Time Bayesian Networks. J. Artif. Int. Res. 39, 1 (sep 2010), 745–774.
on Security and Privacy. IEEE Computer Society, Los Alamitos, CA, USA, 17–31. [86] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni.
https://doi.org/10.1109/SP.2006.26 2019. Modeling Tabular Data Using Conditional GAN. In Proceedings of the
[67] Robert Philipp, Andreas Mladenow, Christine Strauss, and Alexander Völz. 2021. 33rd International Conference on Neural Information Processing Systems. Curran
Machine Learning as a Service: Challenges in Research and Applications. In Associates Inc., Red Hook, NY, USA, Article 659, 11 pages.
Proceedings of the 22nd International Conference on Information Integration and [87] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni.
Web-Based Applications & Services (Chiang Mai, Thailand) (iiWAS ’20). Association 2019. Modeling Tabular Data Using Conditional GAN. In Advances in Neural
for Computing Machinery, New York, NY, USA, 396–406. https://doi.org/10. Information Processing Systems, Vol. 32. Curran Associates, Inc.
1145/3428757.3429152 [88] Kun Yang, Samory Kpotufe, and Nick Feamster. 2021. Feature Extraction for
[68] Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. Novelty Detection in Network Traffic. https://doi.org/10.48550/arXiv.2006.16993
2020. Intriguing Properties of Adversarial ML Attacks in the Problem Space. In arXiv:2006.16993 [cs]
2020 IEEE Symposium on Security and Privacy (SP). IEEE, San Francisco, CA, USA, [89] Limin Yang, Zhi Chen, Jacopo Cortellazzi, Feargus Pendlebury, Kevin Tu, Fabio
1332–1349. https://doi.org/10.1109/SP40000.2020.00073 Pierazzi, Lorenzo Cavallaro, and Gang Wang. 2023. Jigsaw Puzzle: Selective
[69] Babak Rahbarinia, Roberto Perdisci, and Manos Antonakakis. 2015. Segugio: Backdoor Attack to Subvert Malware Classifiers. In IEEE Symposium on Security
Efficient Behavior-Based Tracking of Malware-Control Domains in Large ISP & Privacy.
Networks. In 2015 45th Annual IEEE/IFIP Int’l. Conf. on Dependable Systems and [90] Jim Young, Patrick Graham, and Richard Penny. 2009. Using Bayesian Networks
Networks. IEEE, 403–414. to Create Synthetic Data. Journal of Official Statistics 25 (12 2009), 549–567.
[70] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic [91] Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y. Chen. 2021. CTAB-GAN:
Backpropagation and Approximate Inference in Deep Generative Models. In Effective Table Data Synthesizing. In Proceedings of The 13th Asian Conference on
Proceedings of the 31st International Conference on International Conference on Ma- Machine Learning (Proceedings of Machine Learning Research, Vol. 157), Vineeth N.
chine Learning - Volume 32 (Beijing, China) (ICML’14). JMLR.org, II–1278–II–1286. Balasubramanian and Ivor Tsang (Eds.). PMLR, 97–112. https://proceedings.mlr.
[71] Mauro Ribeiro, Katarina Grolinger, and Miriam A.M. Capretz. 2015. MLaaS: press/v157/zhao21a.html
Machine Learning as a Service. In 2015 IEEE 14th International Conference on

349
ACSAC ’23, December 04–08, 2023, Austin, TX, USA Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, and Jason Matterer

A DATA FORMAT source or destination of the connection. “OTHER” is a placeholder

In Table 5, we illustrate the data format of connection logs extracted for destination ports that are not among the selected 17 ports.
with the Zeek utility from PCAP network data. This represents the The table highlights the importance of features related to port
problem-space format in our setting. 25, which is the port used by the Neris botnet. Commonly se-
lected features for port 25 relate to the number of packets ex-
Table 5: Network data format. Our data is represented by changed (pkts_out_sum_s_25), number of connections in state S05
connection logs (“conn.log” files) extracted with the Zeek (state_S0_s_25), count of distinct external IPs communicating on
monitoring tool from publicly-available packet-level PCAP this port (distinct_external_ips_s_25), and the number of TCP con-
files. nections (tcp_count_s_25). Features related to the connection state
RSTRH6 are also common, although often the value assigned is 0.
Since these are count-based features, a value of zero means this
Name Description
state did not occur within the time window.
orig_ip, resp_ip Source and destination IP address
orig_p, resp_p Source and destination port Table 7: Top 20 most frequently selected features for the
proto Transport Protocol (e.g., TCP, UDP, or ICMP) experiments on CTU-13 Neris.
service Application protocol (e.g., ssh, dns, etc.)
ts Timestamp – the connection start time state_RSTRH_d_443 bytes_in_max_d_1
duration Duration of connection state_RSTRH_d_161 pkts_out_sum_s_25
orig_pkts, resp_pkts Number of transmitted packets state_RSTRH_d_445 state_S0_s_OTHER
orig_bytes, resp_bytes Number of payload bytes state_RSTRH_d_138 state_RSTR_d_OTHER
conn_state Connection state, assessing whether the state_RSTRH_d_135 pkts_out_max_s_OTHER
connection was established and terminated state_S0_s_80 state_S0_s_25
normally (13 different states) state_RSTRH_s_OTHER state_SHR_s_80
tcp_count_s_OTHER tcp_count_s_25
state_REJ_s_443 state_RSTRH_d_123
In Table 6, we illustrate the statistical feature representation
distinct_external_ips_s_25 pkts_in_sum_s_OTHER
used in this work. The statistical features are aggregated over log
connections, which are partitioned by timestamp, internal IP and
destination port. C TRIGGER EXAMPLE
Here, we describe an example of a trigger pattern created with
Table 6: Statistical features aggregated over connection logs
our attack strategy (Section 4). First, the top eight most important
within each data point grouping. The grouping is comprised
features are selected (column 1 in Table 8) using the Entropy feature
of connections within 30-sec time windows, aggregated sepa-
selection method. Next, we compute value assignments and look for
rately for each internal IP and destination port within the
a feature-space prototype, shown in Table 8. Lastly, this prototype
time window. Note that the internal IP versus external IP
is mapped to a set of actual network connections. Table 9 shows
distinction pertains to the subnet, not to the two ends of the
a subset of 10 connection events (the entire pattern consists of 54
connection (source/destination).
connections) from the final trigger pattern.

Field Description Table 8: Example of a trigger prototype in feature space, show-

Aggregation Key: ing only the set of selected features. Entropy selection strat-
(time window, internal IP, destination port) egy on the CTU-13 Neris data.
proto Count of connections per transport protocol
conn_state Count of connections for each conn_state Feature Value
orig_pkts, resp_pkts Sum, min, max over packets pkts_out_sum_s_25 62
orig_bytes, resp_bytes Sum, min, max over bytes tcp_count_s_OTHER 12
duration Sum, min, max over duration state_S3_s_OTHER 0
Aggregation Key: (time window, internal IP) state_RSTRH_s_OTHER 0
ip Count of distinct external IPs state_SHR_s_80 0
state_RSTRH_d_161 0
resp_p Count of distinct destination ports
state_RSTRH_d_138 0
state_RSTRH_d_135 0

B SELECTED FEATURES
While the features selected to form the trigger will change accord- D MODELING THE BYTES DISTRIBUTION
ing to selection strategy, victim model, and randomness effects, we In Figure 7, we present the modeling of two log field values using
report in Table 7 the list of 20 most frequently selected features the Kernel Density Estimation (KDE): responder bytes (left side)
in our experiments on CTU-13 Neris. Since our features are aggre- 5 S0:
Connection attempt seen, no reply observed by Zeek.
gated by internal IP (in addition to time window and port), in the 6 RSTRH: The responder sent a SYN ACK and then a reset, while Zeek did not observe
table, “_s_” and “_d_” distinguish between the internal IP being the a SYN from the originator.

350
Poisoning Network Flow Classifiers ACSAC ’23, December 04–08, 2023, Austin, TX, USA

Table 9: Excerpt of 10 consecutive connection events from a trigger. Due to space constraints, only the relevant fields are shown.

id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state orig_pkts resp_pkts
147.32.84.165 2293 67.195.168.230 25 tcp - 3.004297 0 0 S0 2 0
147.32.84.165 2297 168.95.5.51 25 tcp - 3.004231 0 0 S0 2 0
147.32.84.165 2298 67.195.168.230 25 tcp - 2.987467 0 0 S0 2 0
147.32.84.165 2303 74.125.113.27 25 tcp - 2.987476 0 0 S0 2 0
147.32.84.165 2359 174.133.57.141 80 tcp http 0.310870 358 1765 SF 6 5
147.32.84.165 2367 31.192.109.167 80 tcp http 3.170172 229 194 SF 6 5
147.32.84.165 2368 174.133.57.141 80 tcp http 124.109969 358 2920 RSTO 6 5
147.32.84.165 2354 212.117.174.7 4506 tcp - 3.009544 0 0 S0 2 0
147.32.84.165 2353 212.117.171.138 65500 tcp - 57.278569 883 1097 SF 23 24

(a) Mutual information on clean data, computed on the adversary’s

dataset.

Figure 7: Modeling the bytes distribution for responder (left

side) and originator (right side): From top to bottom, the
figures show: distribution of byte counts per packet, learned
KDEs, and sampled data from the learned distributions.

and originator bytes (right bytes). The figure shows the observed
distribution of bytes per packet in the adversary’s dataset (top row),
and the KDEs distribution of these fields learned from the data
(middle row), and the distribution of the sampled values (bottom
row). Note the similar distribution across the three rows, for each
of the two fields, which indicates that the KDE method is able to
capture the data distribution well. (b) Mutual information on the poisoned training dataset

E MUTUAL INFORMATION Figure 8: Mutual information comparison on clean and poi-

We compare the normalized mutual information on a clean dataset soned data. Showing associations between relevant fields of
(Figure 8a) and on a poisoned dataset (Figure 8b), to show that the conn.log file for CTU-13.
the field-level relationships are generally preserved. Note that we
are interested in maintaining the association patterns, and not the
actual coefficient values. The mutual information statistic is used
in conjunction with the correlation matrix and domain knowledge
to design the direct acyclic graph.

351