Computer Science and Information Systems 2022 Volume 19, Issue 1, Pages: 87-116
https://doi.org/10.2298/CSIS201229045I
Full text (
1057 KB)
Cited by
Entropy-based network traffic anomaly classification method resilient to deception
Ibrahim Juma 
(University of Belgrade, School of Electrical Engineering, Belgrade, Serbia), jumaibrahim04@yahoo.com
Gajin Slavko (University of Belgrade, School of Electrical Engineering, Belgrade, Serbia), slavko.gajin@rcub.bg.ac.rs
Entropy-based network traffic anomaly detection techniques are attractive due to their simplicity and applicability in a real-time network environment. Even though flow data provide only a basic set of information about network communications, they are suitable for efficient entropy-based anomaly detection techniques. However, a recent work reported a serious weakness of the general entropy-based anomaly detection related to its susceptibility to deception by adding spoofed data that camouflage the anomaly. Moreover, techniques for further classification of the anomalies mostly rely on machine learning, which involves additional complexity. We address these issues by providing two novel approaches. Firstly, we propose an efficient protection mechanism against entropy deception, which is based on the analysis of changes in different entropy types, namely Shannon, Rényi, and Tsallis entropies, and monitoring the number of distinct elements in a feature distribution as a new detection metric. The proposed approach makes the entropy techniques more reliable. Secondly, we have extended the existing entropy-based anomaly detection approach with the anomaly classification method. Based on a multivariate analysis of the entropy changes of multiple features as well as aggregation by complex feature combinations, entropy-based anomaly classification rules were proposed and successfully verified through experiments. Experimental results are provided to validate the feasibility of the proposed approach for practical implementation of efficient anomaly detection and classification method in the general real-life network environment.
Keywords: anomaly classification, anomaly detection, entropy, entropy deception, network behaviour analysis
Show references
Ј. Mazel, R. Fontugne, K. Fukuda, A taxonomy of anomalies in backbone network traffic, in: Proceedings of the 2014 International Wireless Communications and Mobile Computing Conference (IWCMC), Nicosia, Cyprus, 4-8 Aug 2014: 30-36. IEEE. doi: 10.1109/IWCMC.2014.6906328.
G. Nychis, V. Sekar, D.G. Andersen, H. Kim, H. Zhang, An Empirical Evaluation of Entropy-based Traffic Anomaly Detection, in: Proceedings of the 8th ACM SIGCOMM Conference on Internet Measurement (IMC “08), Vouliagmeni, Greece, 20-22 October 2008: 151-156. ACM New York, NY, USA. doi: 10.1145/1452520.1452539.
B. Tellenbach, M. Burkhart, D. Schatzmann, D. Gugelmanna, D. Sornette, Accurate network anomaly classification with generalized entropy metrics, Computer Networks 55 (11), (2011) 3485-3502, doi: 10.1016/j.comnet.2011.07.008.
P. Berezinski, B. Jasiul, M. Szpyrka, An entropy-based network anomaly detection method, Entropy 17 (4): 2367-2408, (2015) doi: doi.org/10.3390/e17042367.
I. Özçelik, R. R. Brooks, Deceiving entropy based DoS detection, Computers & Security 48, (2015) 234-245, doi: 10.1016/j.cose.2014.10.013.
B. Claise, Cisco Systems NetFlow Services Export Version 9, RFC 3954.
B. Li, J. Springer, G. Bebis, M.H. Gunes, A survey of network flow applications, Journal of Network and Computer Applications 36 (2), (2013) 567-581. doi: 10.1016/j.jnca.2012.12.020.
M. Ahmed, A.N. Mahmood, J. Hu, A survey of network anomaly detection techniques, Journal of Network and Computer Applications 60, (2016) 19-31. doi: 10.1016/j.jnca.2015.11.016.
V. Chandola, A. Banerjee, V. Kumar, Anomaly Detection: A Survey, ACM Computing Surveys 41 (3), (2009), doi: 10.1145/1541880.1541882.
N. Moustafa, J. Hu, J. Slay, A holistic review of Network Anomaly Detection Systems: A comprehensive survey, Journal of Network and Computer Applications 128, (2019) 33-55. doi: 10.1016/j.jnca.2018.12.006.
M.F. Umer, M. Fahad, M. Sher, Y. Bi, Flow-based intrusion detection: Techniques and challenges, Computers & Security 70, (2017) 238-254. doi: 10.1016/j.cose.2017.05.009.
A. Sperotto, G. Schaffrath, R. Sadre, C. Morariu, A. Pras, B. Stiller, An Overview of IP Flow-Based Intrusion Detection, IEEE Communications Surveys & Tutorials 12 (3), (2010) 343 - 356. doi: 10.1109/SURV.2010.032210.00054.
C.E. Shannon, A mathematical theory of communication, Bell system technical journal, 27(3), 1948, 379-423, doi: 10.1002/j.1538-7305.1948.tb01338.x.
N. Moustafa, G. Creech, J. Slay, Flow Aggregator Module for Analysing Network Traffic, Progress in Computing, Analytics and Networking, Advances in Intelligent Systems and Computing, vol. 710 (2018) 19-29. Springer, Singapore. doi: 10.1007/978-981-10-7871-2_3.
A. Lakhina, M. Crovella, C. Diot, Diagnosing Network-Wide Traffic Anomalies, ACM SIGCOMM Computer Communication Review 34 (4), (2004) 219-230. doi: 10.1145/1030194.1015492.
P.D. Bojovic, I. Basicevic, S. Ocovaj, M. Popovic, A practical approach to detection of distributed denial-of-service attacks using a hybrid detection method, Computers and Electrical Engineering 73, (2018) 84-96. doi: 10.1016/j.compeleceng.2018.11.004.
O. Joldzic, Z. Djuric, P. Vuletic, A transparent and scalable anomaly-based DoS detection method, Computer Networks 104, (2016) 27-42. doi: 10.1016/j.comnet.2016.05.004.
D. Roosi, S. Valenti, Fine-grained traffic classification with netflow data, in: Proceedings of the 6th International Wireless Communications and Mobile Computing Conference, Caen, France, June 28 - July 02 2010: 479-483. ACM New York, NY, USA. doi: 10.1145/1815396.1815507.
P. Barford, J. Kline, D. Plonka, A. Ron, A signal analysis of network traffic anomalies, in: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurement, Marseille, France, November 06 - 08 2002: 71-82. doi: 10.1145/637201.637210.
H.A. Nguyen, T. Van Nguyen, D.I. Kim, D. Choi, Network Traffic Anomalies Detection and Identification with Flow Monitorin, 5th IFIP International Conference on Wireless and Optical Communications Networks (WOCN '08), Surabaya, Indonesia, 5-7 May 2008. IEEE. doi: 10.1109/WOCN.2008.4542524.
R. Braga, E. Mota, A. Passito, Lightweight DDoS flooding attack detection using NOX/OpenFlow, IEEE 35th Conference on Local Computer Networks, Denver, USA, 10-14 Oct. 2010. IEEE: 408-415. DOI: 10.1109/LCN.2010.5735752.
Y. Feng, R. Guo, D. Wang, B. Zhang, Research on the Active DDoS Filtering Algorithm Based on IP Flow, 2009 Fifth International Conference on Natural Computation. Tianjin, China, 14-16 Aug. 2009: 628-632. IEEE.
A. Lakhina, M. Crovella, C. Diot, Mining Anomalies Using Traffic Feature Distributions, Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications 35 (4), (2005) 217-228. doi: 10.1145/1080091.1080118.
T. Pevný, M. Rehák, M. Grill, Identifying suspicious users in corporate networks, Proceedings of workshop on information forensics and security: 1-6, (2012).
C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, Journal of Statistical Physics 52 (1-2), (1988) 479-487, doi: 10.1007/BF01016429.
K. Xu, Z.L. Zhang, S. Bhattacharyya, Internet traffic behaviour profiling for network security monitoring, IEEE/ACM Transactions on Networking 16 (6), (2008) 1241-1252, doi: 10.1109/TNET.2007.911438.
A. Rényi, On measures of entropy and information, in: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability 1, (1961) 547-561.
K. Giotis, C. Argyropoulos, G. Androulidakis, D. Kalogeras, V.Maglaris, Combining OpenFlow and sFlow for an effective and scalable anomaly detection and mitigation mechanism on SDN environments, Computer Networks, Vol 62, (2014), 122-136, doi: 10.1016/j.bjp.2013.10.014
J. Ibrahim, V. Timčenko, S. Gajin, A comprehensive flow-based anomaly detection architecture using entropy calculation and machine learning classification, Proceedings of the 9th International Conference on Information Society and Technology, ISBN 978-86-85525-24-7, (2019) 138-143
V. Timčenko, S. Gajin, Time-series entropy data clustering for effective anomaly detection, Proceedings of the 10th International Conference on Information Society and Technology, Information Society of Serbia, ISBN 978-86-85525-24-7, (2020) 170-175.
R. Sadre, A. Sperotto, A. Pras, The effects of DDoS attacks on flow monitoring applications, 2012 IEEE Network Operations and Management Symposium, (2012) 269-277, doi:10.1109/NOMS.2012.6211908.
L. Ertöz, E. Eilertson, A. Lazarevic, P.N. Tan, V. Kumar, J. Srivastava, P. Dokas, Chapter 3: The MINDS - Minnesota Intrusion Detection System, Next Generation Data Mining, MIT Press, Boston.
C. Fachkha, E. Bou-Harb, M. Debbabi, Fingerprinting Internet DNS Amplification DDoS activities, in: 6th International Conference on New Technologies, Mobility and Security (NTMS), Dubai, United Arab Emirates, 30 March-2 April 2014, 1-5, doi: 10.1109/NTMS.2014.6814019.
A. J. Lawrance, P.A.W. Lewis, An exponential moving-average sequence and point process (EMA1), Journal of Applied Probability 14 (1), (1977) 98-113, doi: 10.2307/3213263.
NetVizura, “NetVizura Netflow Analyzer, Case study - DDoS Attack by NTP Amplification.”, Accessed 22 July 2020. https://www.netvizura.com/files/products/netflow/resources/doc/DDoS-Attack-by-NTP-Amplification-NetVizura.pdf.
M. Allman, V. Paxson, J. Terrell, A brief history of scanning, in: Proceedings of the 7th ACM SIGCOMM conference on Internet measurement, San Diego, California, USA - October 24 - 26, (2007) 77-82. doi: 10.1145/1298306.1298316.
R. Hofstede, L. Hendriks, A. Sperotto, A. Pras, SSH compromise detection using NetFlow/IPFIX, ACM SIGCOMM Computer Communication Review 44 (5): 20-26. ACM New York, NY, USA, (2014) doi: 10.1145/2677046.2677050.
I. Sharafaldin, A.H. Lashkari, A. Ghorbani, Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterizatio, in: Proceedings of the 4th International Conference on Information Systems Security and Privacy - Volume 1: ICISSP, ISBN 978-989-758-282-0, (2018) pages 108-116. doi: 10.5220/0006639801080116.
S. Garcia, M. Grill, J. Stiborek, A. Zunino, An empirical comparison of botnet detection methods, Computers and Security Journal 45, (2014) 100-123. doi: 10.1016/j.cose.2014.05.011.