AIセキュリティポータル K Program
KGV: Integrating Large Language Models with Knowledge Graphs for Cyber Threat Intelligence Credibility Assessment
Share
Abstract
Cyber threat intelligence is a critical tool that many organizations and individuals use to protect themselves from sophisticated, organized, persistent, and weaponized cyber attacks. However, few studies have focused on the quality assessment of threat intelligence provided by intelligence platforms, and this work still requires manual analysis by cybersecurity experts. In this paper, we propose a knowledge graph-based verifier, a novel Cyber Threat Intelligence (CTI) quality assessment framework that combines knowledge graphs and Large Language Models (LLMs). Our approach introduces LLMs to automatically extract OSCTI key claims to be verified and utilizes a knowledge graph consisting of paragraphs for fact-checking. This method differs from the traditional way of constructing complex knowledge graphs with entities as nodes. By constructing knowledge graphs with paragraphs as nodes and semantic similarity as edges, it effectively enhances the semantic understanding ability of the model and simplifies labeling requirements. Additionally, to fill the gap in the research field, we created and made public the first dataset for threat intelligence assessment from heterogeneous sources. To the best of our knowledge, this work is the first to create a dataset on threat intelligence reliability verification, providing a reference for future research. Experimental results show that KGV (Knowledge Graph Verifier) significantly improves the performance of LLMs in intelligence quality assessment. Compared with traditional methods, we reduce a large amount of data annotation while the model still exhibits strong reasoning capabilities. Finally, our method can achieve XXX accuracy in network threat assessment.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
Published: 2019
Few-Shot Learning of TTPs Classification Using Large Language Models
Yu Fengrui, Yanhui Du
Published: 2024
Enabling efficient cyber threat hunting with cyber threat intelligence
P. Gao, F. Shao, X. Liu, X. Xiao, Z. Qin, F. Xu, P. Mittal, S. R. Kulkarni, D. Song
Published: 2021
Quality evaluation of cyber threat intelligence feeds
Harm Griffioen, Tim Booij, Christian Doerr
Published: 2020
Are you talking to me? a case study in emotional human-machine interaction
Manuel Flurin Hendry, et al.
Published: 2023
Llm-tikg: Threat intelligence knowledge graph construction utilizing large language model
Yuelin Hu, et al.
Published: 2024
Ttpdrill: Automatic and accurate extraction of threat actions from unstructured text of cti sources
G. Husari, E. Al-Shaer, M. Ahmed, B. Chu, X. Niu
Published: 2017
Towards fine-grained reasoning for fake news detection
Yiqiao Jin, et al.
Published: 2022
Cyber threat intelligence
Martin Lee
Published: 2023
Attackg: Constructing technique knowledge graph from cyber threat intelligence reports
Z. Li, J. Zeng, Y. Chen, Z. Liang
Published: 2022
Correlation of cyber threat intelligence with sightings for intelligence assessment and augmentation
Po-Ching Lin, et al.
Published: 2023
Poirot: Aligning attack behavior with kernel audit records for cyber threat hunting
S. M. Milajerdi, B. Eshete, R. Gjomemo, V. Venkatakrishnan
Published: 2019
LLM Driven Web Profile Extraction for Identical Names
Prateek Sancheti, Kamalakar Karlapalem, Kavita Vemuri
Published: 2024
Casie: Extracting cybersecurity event information from text
Taneeya Satyapanich, Francis Ferraro, Tim Finin
Published: 2020
FakeKG: a knowledge graph of fake claims for improving automated fact-checking (student abstract)
Gautam Kishore Shahi
Published: 2023
Time for aCTIon: Automated Analysis of Cyber Threat Intelligence in the Wild
Giuseppe Siracusano, Davide Sanvito, Roberto Gonzalez, Manikantan Srinivasan, Sivakaman Kamatchi, Wataru Takahashi, Masaru Kawakita, Takahiro Kakumaru, Roberto Bifulco
Published: 2023.7.14
What’s in a cyber threat intelligence sharing platform? A mixed-methods user experience investigation of MISP
Borce Stojkovski, et al.
Published: 2021
Sequence level contrastive learning for text summarization
Shusheng Xu, et al.
Published: 2022
Share