AIセキュリティポータル K Program
IT Intrusion Detection Using Statistical Learning and Testbed Measurements
Share
Abstract
We study automated intrusion detection in an IT infrastructure, specifically the problem of identifying the start of an attack, the type of attack, and the sequence of actions an attacker takes, based on continuous measurements from the infrastructure. We apply statistical learning methods, including Hidden Markov Model (HMM), Long Short-Term Memory (LSTM), and Random Forest Classifier (RFC) to map sequences of observations to sequences of predicted attack actions. In contrast to most related research, we have abundant data to train the models and evaluate their predictive power. The data comes from traces we generate on an in-house testbed where we run attacks against an emulated IT infrastructure. Central to our work is a machine-learning pipeline that maps measurements from a high-dimensional observation space to a space of low dimensionality or to a small set of observation symbols. Investigating intrusions in offline as well as online scenarios, we find that both HMM and LSTM can be effective in predicting attack start time, attack type, and attack actions. If sufficient training data is available, LSTM achieves higher prediction accuracy than HMM. HMM, on the other hand, requires less computational resources and less training data for effective prediction. Also, we find that the methods we study benefit from data produced by traditional intrusion detection systems like SNORT.
Snort - lightweight intrusion detection for networks
M. Roesch
Published: 1999
Survey of intrusion detection systems: techniques, datasets, and challenges
Khraisat, A., et al.
Published: 2019
Learning near-optimal intrusion responses against dynamic attackers
K. Hammar, R. Stadler
Published: 2023
Learning security strategies through game play and optimal stopping
K. Hammar, R. Stadler
Published: 2022
The viterbi algorithm
G. D. Forney
Published: 1973
Credit card fraud detection using hidden markov model
A. Srivastava, A. Kundu, S. Sural, A. Majumdar
Published: 2008
Using hidden markov models to evaluate the risks of intrusions: System architecture and model validation
A. ARNES, F. VALEUR, G. VIGNA, R. A. KEMMERER
Published: 2006
The application of baum-welch algorithm in multistep attack
Y. Zhang, D. Zhao, J. Liu
Published: 2014
Hidden markov models and alert correlations for the prediction of advanced persistent threats
I. Ghafir, K. G. Kyriakopoulos, S. Lambotharan, F. J. Aparicio-Navarro, B. AsSadhan, H. Binsalleeh, D. M. Diab
Published: 2019
Real-time multistep attack prediction based on hidden markov models
P. Holgado, V. A. Villagra, L. Vazquez
Published: 2017
A tutorial on hidden Markov models and selected applications in speech recognition
L.R. Rabiner
Published: 1989
An introduction to hidden markov models
L. Rabiner, B. Juang
Published: 1986
Long short-term memory
S. Hochreiter, J. Schmidhuber
Published: 1997
Shuffling recurrent neural networks
M. Rotman, L. Wolf
Published: 2021
Random forests
L. Breiman
Published: 2001
Toward generating a new intrusion detection dataset and intrusion traffic characterization
Iman Sharafaldin, Arash Habibi Lashkari, Ali A Ghorbani
Published: 2018
Hidden markov models for malware classification
C. Annachhatre, T. H. Austin, M. Stamp
Published: 2015
Software abnormal behavior detection based on hidden markov model
J. Zhao, G. Huang, T. Liu, B. Cui
Published: 2017
Modeling program behaviors by hidden markov models for intrusion detection
W. Wang, X.-H. Guan, X.-L. Zhang
Published: 2004
Hidden markov model modeling of ssh brute-force attacks
A. Sperotto, R. Sadre, P.-T. de Boer, A. Pras
Published: 2009
A hidden markov model based framework for tracking and predicting of attack intention
X. Zan, F. Gao, J. Han, Y. Sun
Published: 2009
Anomaly network intrusion detection using hidden markov model
C.-M. Chen, D.-J. Guan, Y.-Z. Huang, Y.-H. Ou
Published: 2016
Cloud security based attack detection using transductive learning integrated with hidden markov model
Y. Aoudni, C. Donald, A. Farouk, K. B. Sahay, D. V. Babu, V. Tripathi, D. Dhabliya
Published: 2022
A novel intrusions detection method based on hmm embedded neural network
W. Jiang, Y. Xu, Y. Xu
Published: 2005
Applying long short-term memory recurrent neural networks to intrusion detection
R. C. Staudemeyer
Published: 2015
Leveraging lstm networks for attack detection in fog-to-things communications
A. Diro, N. Chilamkurti
Published: 2018
Lstm for anomaly-based network intrusion detection
S. A. Althubiti, E. M. Jones, K. Roy
Published: 2018
Intrusion detection systems using long short-term memory (lstm)
F. Laghrissi, S. Douzi, K. Douzi, B. Hssina
Published: 2021
Advance persistent threat detection using long short term memory (lstm) neural networks
P. Sai Charan, T. Gireesh Kumar, P. Mohan Anand
Published: 2019
Detecting multi-stage attacks using sequence-to-sequence model
P. Zhou, G. Zhou, D. Wu, M. Fei
Published: 2021
A framework for fast and efficient cyber security network intrusion detection using apache spark
G. P. Gupta, M. Kulariya
Published: 2016
Performance evaluation of supervised machine learning algorithms for intrusion detection
M. C. Belavagi, B. Muniyal
Published: 2016
Network attribute selection, classification and accuracy (nasca) procedure for intrusion detection systems
Z. Stefanova, K. Ramachandran
Published: 2017
Detecting mobile botnets through machine learning and system calls analysis
V. G. da Costa, S. Barbon, R. S. Miani, J. J. Rodrigues, B. B. Zarpelao
Published: 2017
Characterization of tor traffic using time based features
A. Lashkari, G. Gil, M. Mamun, A. Ghorbani
Published: 2017
Active learning intrusion detection using k-means clustering selection
S. McElwee
Published: 2017
A simple and robust approach of random forest for intrusion detection system in cyber security
M. Choubisa, R. Doshi, N. Khatri, K. K. Hiran
Published: 2022
Share