Cyber threat intelligence is a critical tool that many organizations and
individuals use to protect themselves from sophisticated, organized,
persistent, and weaponized cyber attacks. However, few studies have focused on
the quality assessment of threat intelligence provided by intelligence
platforms, and this work still requires manual analysis by cybersecurity
experts. In this paper, we propose a knowledge graph-based verifier, a novel
Cyber Threat Intelligence (CTI) quality assessment framework that combines
knowledge graphs and Large Language Models (LLMs). Our approach introduces LLMs
to automatically extract OSCTI key claims to be verified and utilizes a
knowledge graph consisting of paragraphs for fact-checking. This method differs
from the traditional way of constructing complex knowledge graphs with entities
as nodes. By constructing knowledge graphs with paragraphs as nodes and
semantic similarity as edges, it effectively enhances the semantic
understanding ability of the model and simplifies labeling requirements.
Additionally, to fill the gap in the research field, we created and made public
the first dataset for threat intelligence assessment from heterogeneous
sources. To the best of our knowledge, this work is the first to create a
dataset on threat intelligence reliability verification, providing a reference
for future research. Experimental results show that KGV (Knowledge Graph
Verifier) significantly improves the performance of LLMs in intelligence
quality assessment. Compared with traditional methods, we reduce a large amount
of data annotation while the model still exhibits strong reasoning
capabilities. Finally, our method can achieve XXX accuracy in network threat
assessment.
外部データセット
CTI-200
参考文献
Proceedings of NAACL-HLT
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
Published: 2019
Few-Shot Learning of TTPs Classification Using Large Language Models
Yu Fengrui, Yanhui Du
Published: 2024
2021 IEEE 37th International Conference on Data Engineering (ICDE)
Enabling efficient cyber threat hunting with cyber threat intelligence
P. Gao, F. Shao, X. Liu, X. Xiao, Z. Qin, F. Xu, P. Mittal, S. R. Kulkarni, D. Song
Published: 2021
Applied Cryptography and Network Security: 18th International Conference, ACNS 2020
Quality evaluation of cyber threat intelligence feeds
Harm Griffioen, Tim Booij, Christian Doerr
Published: 2020
Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Are you talking to me? a case study in emotional human-machine interaction
Manuel Flurin Hendry, et al.
Published: 2023
Computers & Security
Llm-tikg: Threat intelligence knowledge graph construction utilizing large language model
Yuelin Hu, et al.
Published: 2024
Proceedings of the 33rd annual computer security applications conference
Ttpdrill: Automatic and accurate extraction of threat actions from unstructured text of cti sources
G. Husari, E. Al-Shaer, M. Ahmed, B. Chu, X. Niu
Published: 2017
Proceedings of the AAAI Conference on Artificial Intelligence
Towards fine-grained reasoning for fake news detection
Yiqiao Jin, et al.
Published: 2022
John Wiley & Sons
Cyber threat intelligence
Martin Lee
Published: 2023
European Symposium on Research in Computer Security
Attackg: Constructing technique knowledge graph from cyber threat intelligence reports
Z. Li, J. Zeng, Y. Chen, Z. Liang
Published: 2022
Computer Networks
Correlation of cyber threat intelligence with sightings for intelligence assessment and augmentation
Po-Ching Lin, et al.
Published: 2023
Proceedings of the 2019 ACM SIGSAC conference on computer and communications security
Poirot: Aligning attack behavior with kernel audit records for cyber threat hunting
S. M. Milajerdi, B. Eshete, R. Gjomemo, V. Venkatakrishnan
Published: 2019
Companion Proceedings of the ACM on Web Conference
LLM Driven Web Profile Extraction for Identical Names