AIセキュリティポータル K Program
On the Cross-Dataset Generalization of Machine Learning for Network Intrusion Detection
Share
Abstract
Network Intrusion Detection Systems (NIDS) are a fundamental tool in cybersecurity. Their ability to generalize across diverse networks is a critical factor in their effectiveness and a prerequisite for real-world applications. In this study, we conduct a comprehensive analysis on the generalization of machine-learning-based NIDS through an extensive experimentation in a cross-dataset framework. We employ four machine learning classifiers and utilize four datasets acquired from different networks: CIC-IDS-2017, CSE-CIC-IDS2018, LycoS-IDS2017, and LycoS-Unicas-IDS2018. Notably, the last dataset is a novel contribution, where we apply corrections based on LycoS-IDS2017 to the well-known CSE-CIC-IDS2018 dataset. The results show nearly perfect classification performance when the models are trained and tested on the same dataset. However, when training and testing the models in a cross-dataset fashion, the classification accuracy is largely commensurate with random chance except for a few combinations of attacks and datasets. We employ data visualization techniques in order to provide valuable insights on the patterns in the data. Our analysis unveils the presence of anomalies in the data that directly hinder the classifiers capability to generalize the learned knowledge to new scenarios. This study enhances our comprehension of the generalization capabilities of machine-learning-based NIDS, highlighting the significance of acknowledging data heterogeneity.
Cyber risk and cybersecurity: a systematic review of data availability
F. Cremer, B. Sheehan, M. Fortmann, A. N. Kia, M. Mullins, F. Murphy, S. Materne
Published: 2022
Machine learning techniques applied to cybersecurity
J. Mart´ınez Torres, C. Iglesias Comesana, P. J. Garc ˜ ´ıa-Nieto
Published: 2019
Machine learning in cybersecurity: a comprehensive survey
D. Dasgupta, Z. Akhtar, S. Sen
Published: 2022
Cybersecurity data science: an overview from machine learning perspective
I. H. Sarker, A. Kayes, S. Badsha, H. Alqahtani, P. Watters, A. Ng
Published: 2020
Network anomaly detection: methods, systems and tools
M. H. Bhuyan, D. K. Bhattacharyya, J. K. Kalita
Published: 2013
Intrusion detection systems: A state-of-the-art taxonomy and survey
M. Alkasassbeh, S. Al-Haj Baddar
Published: 2023
Survey of intrusion detection systems: techniques, datasets, and challenges
Khraisat, A., et al.
Published: 2019
A system for denial-of-service attack detection based on multivariate correlation analysis
Z. Tan, A. Jamdagni, X. He, P. Nanda, R. P. Liu
Published: 2013
Pca-based multivariate statistical network monitoring for anomaly detection
J. Camacho, A. Perez-Villegas, P. Garc ´ ´ıa-Teodoro, G. Macia-´ Fernandez
Published: 2016
A secure intrusion detection system for manets
R. SakilaAnnarasi, S. Sivanesh
Published: 2014
Intrusion signature creation via clustering anomalies
G. R. Hendry, S. J. Yang
Published: 2008
A survey of payload-based traffic classification approaches
M. Finsterbusch, C. Richter, E. Rocha, J.-A. Muller, K. Hanss¬gen
Published: 2013
Data traffic classification in software defined networks (sdn) using supervised-learning
M. M. Raikar, S. Meena, M. M. Mulla, N. S. Shetti, M. Karanandi
Published: 2020
Evaluating standard feature sets towards increased generalisability and explainability of ML-based network intrusion detection
M. Sarhan, S. Layeghy, M. Portmann
Published: 2022
A novel multi-stage approach for hierarchical intrusion detection
M. Verkerken, L. D’hooge, D. Sudyana, Y.-D. Lin, T. Wauters, B. Volckaert, F. De Turck
Published: 2023
Share